Repository navigation
AOT: crashes and wrong answers found since #942, and faster builds of large functions - #967
Conversation
… large functions
Its tests that need Bun are in the Bun PR that goes with this one.
Crashes of compiled programs:
let b = null, c = null;
if (n) c = { k: 1.5 };
while (b !== b) b = [b];
if (y != b) { [b, y] = [y, b]; return c === y ? c : 1; }
ended by SIGSEGV, and ten lines of the same kind aborted the compiler. SinkAllocations analyses a function,
counts and then rewrites it, and all three must decide alike. A helper read constants out of the function as
it stood, so the rewriting saw constants that the analysis had not seen. #942 mended one of its five callers.
Now the question is answered from a table that is filled before anything is rewritten, and that mend is gone.
try { null.x } catch (e) { e.constructor.list = { length: 3 } } // TypeError.list was an array before
ended by a signal, gave `undefined`, or called the function that had been replaced. Flow took the stores that
it could follow for all the stores there are. A store through an object that nobody knows is now looked for on
every built-in class, and on the prototypes above crypto.subtle, Bun.s3 and a request's signal.
class Holder { held = stream[Symbol.for("nodejs.stream.readable")]; } // a ReadableStream
aborted, and in a function the read gave `undefined`. A symbol out of Symbol.for() was treated like one out of
Symbol(), which the host cannot reach.
Wrong answers:
- After `delete A.prototype.twice`, `new A(8).twice()` still answered. With a setter on B.prototype alone,
`const o = k === 0 ? new A(k) : new B(k); o.y = 5` in a loop ran B's setter for the first A. B3 moved the
load of a class's guard bits across the allocation that makes the class.
- `require("./config.json")` in a .cjs file that runs while the graph loads answered the ES namespace.
- `Object.assign({}, source)` stored names before symbols for a source with room for elements.
- Closures of one function that call each other in tail position: a stack frame had the name of the
function object that was called, not of the one that ran.
- With one optional pass off, `o.a` was folded where `o` may be null, and another pass aborted the build.
- Two runs of one executable made different caching decisions: a table was hashed by an address.
Memory: the slow path of a stub has a frame as large as what it saves. Objects were kept alive longer than in
plain Bun. A store of "" or of a string of one Latin-1 character needs no write barrier.
Build time. A generated function of 1,000 statements builds in 7,891 M instructions, from 12,473 M, with the
same output: liveness gave up only after a fixed number of visits and then started over, a dataflow kept a
counter for each condition in each block, and the guards' analysis asked each write about each proof, where
4,304 writes had 28 different answers.
Speed: `a[i] === b[j]`, case conversion sixteen characters at a step, a read by name of a slot that holds a
raw double, and 226 stepped benchmark rows through the stubs' new form.
Checks: the proof checks compare a function's identity and are no longer skipped in forwarding code.
`--aot-check=ClaimsOfFlow` tests every claim of flow where it is made. Under the verifying flags
SinkAllocations checks that its three steps decided alike.
There was a problem hiding this comment.
Beyond the inline findings, I also checked the for (;;) in copyLoopsForFunctionsTheyCall (AOTLoopPasses.cpp) for non-termination — after a copy, the original loop's listed callees all fail findEarlierTest and each copy has a single proven callee, so the search finds nothing more — and the new RELEASE_ASSERT in ProgramIdentifiers::uidSlowCase (AOTProgramIdentifiers.cpp:117), which only guards the VM-shared object of a multi-instance image while realm-created symbols come from the per-realm object, so it is not reached in the multi-realm path.
Extended reasoning...
This is a 538-file, ~18.7k-line change across the JSC AOT compiler, runtime, stubs and tests plus small edits in yarr, WTF and bmalloc, with no auth, crypto or injection surface. Four inline findings were posted; the loop-copying pass termination and the new identifiers assertion were also examined and ruled out as described in the body.
Findings marked 🟡 are optional suggestions and need no follow-up push.
| for (StringID pattern : patterns.strings) | ||
| addCandidates(inputs.program, inputs.context, RegExpRequestOrigin::Candidate, pattern, flagSets, site, found); |
There was a problem hiding this comment.
🟡 (optional) Hard stop removed: nothing now bounds how many predicted RegExp texts one call site compiles into the image, so a build whose pattern texts come from a large list gets an image and compile time that grow with that list, where the base stopped at 64 texts and 64 KiB of code per site. The three limits RegExpCandidatesPerSite, RegExpCandidateCodeBytesPerSite and RegExpCandidateTextLength are deleted from limits.def and every text flow or the constant walk finds is added at AOTRegExpRequests.cpp:3756 and compiled at AOTRegExpCompiler.cpp:541, with only the 2 GiB RegExpSectionSize left as a backstop. Fix: restore a per-site (or per-program) cap on predicted texts and on their code bytes, applied both to flow candidates and to texts from the constant walk, and decline the rest as before.
Why this was flagged
A program whose RegExp text is predicted from many values reaches collectCandidates and addCandidatesFromConstants in AOTRegExpRequests.cpp: a wrapper function re(s) { return new RegExp(s) } called from thousands of sites with distinct literals (AOTValueFlowResultsSinks.cpp:171 collects every literal), or list.map(s => new RegExp(s)) over a data array, whose elements are produced by elementOf at AOTRegExpRequests.cpp:1883-1886 without charging the walk budget. On the base branch takesTextsOfFlow refused a site with more than RegExpCandidatesPerSite (64) texts, add() refused the 65th alternative, and applyCandidateBudget dropped code past RegExpCandidateCodeBytesPerSite (64 KiB) per site. After the change add() at AOTRegExpRequests.cpp:538-542 always accepts, every pattern is appended at AOTRegExpRequests.cpp:3756 and :3877, and compileRegExps at AOTRegExpCompiler.cpp:532-541 compiles each one with applyCandidateBudget removed; the only remaining stop is applySectionSizeLimit at 2 GiB. No code names a bound on the number of predicted texts or their bytes per site or per program.
Verification: limits.def deletes RegExpCandidateCodeBytesPerSite (65536), RegExpCandidatesPerSite (64) and RegExpCandidateTextLength. AOTRegExpCompiler.cpp removes applyCandidateBudget entirely, and compileRegExps now goes straight to applySectionSizeLimit, whose only bound is 2 GiB (AOTRegExpCompiler.cpp:456). collectCandidates now adds every flow string unconditionally at AOTRegExpRequests.cpp:3755-3756.
| // The walk for a caller of the function that holds the call is a walk in its own right: there are as many as the program has such calls, and each is as small as what that caller | ||
| // passes. The chains above one caller share its budget: there can be as many of them as there are ways through the program. | ||
| if (chain.size() == 1) | ||
| m_valuesFollowed = 0; |
There was a problem hiding this comment.
🟡 (optional) Builds of programs with a RegExp helper that has many direct callers now spend up to callers x 16384 value walks on one site, where the base spent one budget per site. AOTRegExpRequests.cpp:385-386 resets m_valuesFollowed to 0 for every direct caller at chain depth 1, and the base's m_worlds counter (bounded by maxAlternatives) that capped the number of worlds explored per site was removed. Fix: restore a per-site total on walks (or a cap on the number of callers walked) so build time for one call site stays bounded regardless of caller count.
Why this was flagged
A helper like function re(s, f) { return new RegExp(s, f) } called from K sites, each passing its own constant, enters exploreForEachCaller (AOTRegExpRequests.cpp:375). For each DirectCall at :381 the budget is reset at :385-386 when chain.size() == 1, so the m_valuesFollowed >= m_budget cut at :387 never fires across callers; each caller may consume the full maxValuesFollowed (1 << 14, :254) and recursive chains above it. The base (saved copy at :161, :315) reset the budget once in patternsOf and additionally stopped after maxAlternatives() worlds via m_worlds; both totals are gone (m_worlds removed in the diff). This entry was dismissed as covered by the candidate at AOTRegExpCompiler.cpp:541, but that candidate was REFUTED, so no filed finding addresses the per-caller walk cost; the CONFIRMED finding at AOTRegExpRequests.cpp:3756 is about how many predicted texts get compiled, a different mechanism and fix. Every build of a program with a widely used RegExp wrapper pays this.
Verification: Triggering condition: a new RegExp(...) site in a helper with many direct callers. At AOTRegExpRequests.cpp:385-386 if (chain.size() == 1) m_valuesFollowed = 0; runs before the m_valuesFollowed >= m_budget check at :387, so that check can never cut the loop. The base reset m_valuesFollowed = 0 once per site, so per-site work goes from ~16384 walks to (number of direct callers) x 16384.
|
Preview build of 43ccc98: |
…crash
Crashes
- `Bun.gzipSync.list = [1, 2]; Bun[JSON.parse('"gzipSync"')].list = { length: 3 }`
and then `Bun.gzipSync.list.length` segfaulted. So did the same through
`Event.prototype[name]`, names out of `Object.keys(Bun)`, and
`Bun.Glob.prototype` reached by three computed names.
`Object.setPrototypeOf(Object.getPrototypeOf(Bun.s3), { .. })` was not seen.
A read under a computed key, or of `prototype`, off an unknown object in the
program's own code now reaches the host prototype objects. This replaces the
wider rule of the previous commit, which also counted such reads in library
code and cost `await request.json()` its typed parse.
- The compiler died with "Floating point error" on
`while (a == 65536 && i0 < 9) { i0++; c = c > 0 ? d : c; }`.
B3VectorizeLoops divided by the number of memory accesses of a loop that has
none.
- A loop counter beyond int32, read from a property and used to index a typed
array, trapped.
- `RegExp::matchWithImageCode` dereferenced a null image record when jsc runs
its tests with regular expressions compiled ahead of time (six JSTests).
Wrong answers
- `/^(?=(.*))\1$/` said "no match" for a subject with characters outside
Latin-1. The pattern was taken to consume 8-bit characters only: what a
lookahead captures counts once a back reference can consume it.
- `new RegExp("\\B" + text, "u")`: `\B` matched inside a surrogate pair.
- `function make(k) { return k === 0 ? new A(0) : new B(1) }` with a getter on
`B.prototype` alone: `make(0).size` ran `B`'s getter when the first `A` was
made behind a test of both classes.
- `x.f += n` where `f` starts on the prototype: the arm that reads `f` by name
may run a getter, and came back into code that relied on a guard from before
the read.
- `Interface.prototype[Symbol.asyncIterator]` read in a function was a
different function object than `Reflect.get()` returned (node:readline).
- `await import("sass")` in an executable threw "Cannot assign to
Function.prototype.call$0". Programs out of dart2js add such names.
- `function twice(f, n) { return f(f(n)) } twice({}, 1)` quoted `f` instead of
`f(n)` in the error message.
- `import * as ns`; `Reflect.setPrototypeOf(ns, ..)`, `Object.freeze(ns)`,
`"added" in ns`: a guard for compiled objects read a module namespace.
Windows
- `hasChanged |= std::exchange(member, copyOfClass(member)) != member` compares
the old value with itself where the right operand is evaluated first, as on
both Windows targets. Copied classes and closure factories kept the types of
the original there: correct, but less precise.
Memory
- `for (let i = 0; i < n; i++) a.push(f(i))` reserved the whole array up front
even where `f` may throw and end the loop.
Faster
- `let current = first; while (current) current = current.run(this)` with
`first` typed by an interface: objects that have the interface's members but
none of its layouts stay in the typed loop. Effect's run loop: -20%
instructions per turn.
Smaller
- The compiler's rarely run files are built for size: -600 KB.
Tools
- The map and `--aot-report` say "proven" or "missed optimization" where the
final IR knows the receiver's class, instead of "unknown".
- False alarms under `--aot-validate` and `--aot-verify-each-pass`: a field
that holds its own object, what an `await` holds, `Object.keys()` of an
object with a computed key.
The previous commit added a bool at the end of FlowSiteLevel. With the Microsoft ABI the two bit-fields at its start take a unit of their own, so the struct was already 24 bytes there and grew to 28, and the static_assert failed every Windows build. The bool now stands behind the two uint8_t members, in what was padding on every target.
…by a test's own edge `bun test --aot` on NestJS's suite stopped with "Stopped while compiling printMessages .. in PlaceTypeTests". A block's parameters are its explicit arguments followed by the results of the edges that enter it, such as the narrowed value of a type test. Two places indexed an edge's arguments with a parameter's index, which is out of bounds for an edge result. Both now ask incomingValueOf(). An edge's own result counts as proven when a rejoin is planned, and as unknown when the first values of loop variables are looked for. The second commit of this pull request brought this.
…ith a long chain of classes Two fixes for two red tests of oven-sh/bun#47571. The first error at a place costs the same in a longer file Forty places inside one `switch` of 12,000 cases each throw their first TypeError ("undefined is not an object (evaluating 'a.b')"). To quote the source in such a message the run time decodes the entries that say what was printed in place of the original text. It decoded every entry of the kept piece of source at each first error, and a long statement or function is one piece. The entries of a piece are now decoded once and kept, however many places in it throw. A compiled test run decoded every entry of the module each time: now once. The long file decodes 2,162 entries where it decoded about 357,000. Forty places in one statement: 2.3 > 1.5 ms. In a test run: 4.05 > 0.59 ms. With AddressSanitizer it was 130 to 141 ms against 2 to 3 ms in the short file, which is what failed the-first-error-at-a-place-costs-the-same-in-a-longer-file on the x64 ASAN lane. No message changes. The counts file has a new line, `decoded:TextPrintedInPlace`. A build keeps the partition of a call's receivers as words `function ask_i(x) { return x.weigh() + ask_(i-1)(x) }`, 800 in a chain, each given an instance of one class more. No two calls of `weigh` see the same classes, so the cache kept one partition per call and key, each with an arm per class: 320,000 arms of 112 bytes. The cache now keeps a partition packed: 24 bytes for an arm with one method and one layout. And "may this call enter a table look-up" built and kept a whole partition to answer one bool: it now asks class by class, stops at the first no and keeps nothing. 800 classes: 189 > 166 MB, 20.14 > 20.01 G instructions. The generated code is identical on 1.16 million functions. build-scaling-4 failed on macOS arm64 with "twice as many classes cost 2.58 times the peakBytes; allowed: 2.5". On Linux that ratio goes from 2.29 to 1.85. Not measured on macOS.
| for (const TextThatWaits& text : waiting) { | ||
| if (comesToEarlierCall && text.waitsApart) | ||
| levelsAddedToPlace = std::max<uint32_t>(levelsAddedToPlace, (text.place >> 33) - state.turnOfTexts); |
There was a problem hiding this comment.
🟡 nit (optional): Programs whose dynamic imports wait for text on another thread can see an import() settle ahead of an earlier call it shares modules with, which the code says must not happen. AOTModules.cpp:2399 computes (text.place >> 33) - state.turnOfTexts in unsigned arithmetic; once an earlier waiting-apart call's turn has already passed (several calls were queued in one turn and are delivered one per turn), the difference wraps to about 2^32 and std::max<uint32_t> picks it over a genuinely later call's level count. Fix: only add a level count when (text.place >> 33) > state.turnOfTexts (saturate at 0), so the max reflects the latest earlier call rather than a wrapped value.
Why this was flagged
Trigger: under bun run, several import() calls are held in one turn with waitsApart set at AOTModules.cpp:2380; deliverTextThatWaits at AOTModules.cpp:2479-2495 delivers one per turn and increments turnOfTexts each time, so a call X held at turn 10 (place >> 33 == 11) can still be waiting at turn 13. A new import() W that comesToEarlierCall and also shares modules with a call Y held at turn 13 with levelsAddedToPlace 2 (place >> 33 == 15) then runs the loop at AOTModules.cpp:2397-2400. For X the subtraction (11) - 13 is uint64 and wraps; converted to uint32 by std::max<uint32_t> it becomes 0xFFFFFFFE, which beats Y's 2, so at AOTModules.cpp:2401 place += 0xFFFFFFFE << 33 wraps mod 2^64 and W lands at turn level 11 instead of 15. W is therefore delivered before Y, although the comment at AOTModules.cpp:2396 states 'What it shares with them comes no sooner for it than for them'. The base branch had no waits-apart ordering, so this is a new, intended ordering guarantee that the wrap defeats; putCallsThatWaitApartInLine at AOTModules.cpp:2410-2421 subtracts the same wrapped count so no crash results, only a wrong settlement order.
Verification: Trigger: under a host with callInLaterTurn, two or more import() calls are held apart in one turn. state.turnOfTexts is uint32_t (Source/JavaScriptCore/aot/runtime/AOTVMState.h:238) and text.place is uint64_t, so (text.place >> 33) - state.turnOfTexts at Source/JavaScriptCore/aot/runtime/AOTModules.cpp:2399 wraps when the waiting text's turn is behind the current one; std::max<uint32_t> narrows it to ~2^32-k, which beats any genuine small offset from a later call.
Pins oven-sh/WebKit#967 at 43ccc98c356a5d1ff47392dde1ced58b51c5d20c. The build - `var holder = { other: 1 }; const kept = holder.link; holder.link = kept; holder.link(Child, Base)`: `bun build --compile --aot` never ended. So with `const original = A.make; .. A.make = original`. The walk from a name to what it stands for forgot the names it had visited when it went on to a stored member. It now remembers each store it has followed. The previous commit brought this. - The exported variables of a build's module records are looked up on the calling thread again. The previous commit put that on threads, which gained nothing on a large application. - `bun_aot` is built at opt-level "s": Bun is 475 KB smaller. `bun test --aot` - Product code finds only what an executable would find. With `exports.load = name => require(dir + name)` in `src/` or in a package, a test that has it load a file the build didn't see now fails as the executable would. Files of the tests load what they like, native addons load wherever the call stands, and `--aot-allow-code-from-disk` turns it off. - `delete require.cache[require.resolve("../src/state.js")]` and then a written `import("./state.js")` in product code runs the module again. - `--shard=k/n` deals the files by what they reach, so that a shard builds less. Every shard computes the same deal on any machine. Plain `bun test --shard` is unchanged. - The parts of a run are built slowest first. - `silence(console, levels)` and `spyAll(console, wanted)`, where a helper in another file stores or spies under a computed name, no longer fail with "Cannot assign to console.debug". - `--files-from=/dev/stdin` failed every file in a layered run. - One file that let go of its `module` or `require` made the run keep every export of every file. In express that was 838 functions and 78 files. - `--aot-memory-limit` says what it bounds: the build, not the tests that run. Module loading - Followed at build time: `require(__filename.replace("main.cjs", "first.cjs"))`, `replaceAll`, `__filename.slice(0, -8) + "third.cjs"`, `import(import.meta.url.replace(/module\.mjs$/, "ninth.mjs"))`. - `new Worker(file)` in a file loaded from disk by an executable built with `--aot-allow-code-from-disk` died on `globalThis.console = ..`. Wrong answers - `String(f)` in a `.ts` file: `() => <Foo>{}` read `() => {}` and `return<number>a` read `returna`. From the review of the previous commit - A `spyOn()` inside a template string with `${..}` for a name was missed: "Cannot spy on 'total'". - The memory of a worker's children was counted in pages of 4 KB. - A failed write into the program below was ignored. Three of our own tests opened that file read-only, so their writes had been failing silently. - A preload that a test file imports back was scanned twice. - A test that can't make its folder says skipped. Tests - The stage-log test used `O_DIRECT` by number, which is `O_DIRECTORY` on arm64. It listens on a datagram socket now. - A helper asked `.todo` of `test.skipIf(true)`, which failed eight files on the ASAN lane. - A stack-frames program cut its path at the first forward slash, which failed on Windows. - Five `gc/` programs are marked todo on Windows x64. - The thread pool's unit tests ran dozens of builds on a pool sized for one. - The test of how a run's parts are cut asked for the order in which three builds end, which is a race. It asks for the cut. - 113 new files, most of them stored programs compared with plain Bun.
Pins WebKit at 5869a3b791e68fd8e0e8059ff21e278760bcf4c6 (oven-sh/WebKit#967, merged) in place of the preview build. Five files conflicted with #44833 and #47070. In all five, main's lines stand as written. Fixes for what this branch brought - `bun test --aot --shard=k/n`: each shard dealt the test files by, among other things, its own `--parallel`. Two runners started with different `--parallel` computed different deals. With `--parallel=1` on one and `--parallel=16` on the other, 21 of 40 test files ran on neither, and both runs reported success. Found in review of this PR. Every shard now computes the deal from what all of them share: the paths, the sizes, what the files reach, and the timings file. How many files run at once is read from the timings file (`"workers"`, written by a compiled `--parallel` run under `--update-timings`). `--parallel` has no say. Each shard prints one dim line with a hash of the whole deal, so a mismatch can be seen. - 10,000 functions that call one another in a ring, each handing on what it got: `bun build --compile --aot` overflowed the stack under AddressSanitizer, and `bun test --aot` did so in release builds. The walk that finds which values a name may stand for recurses once per function of the ring. Where the stack runs low it now goes on in a thread with a fresh stack. 100,000 functions build. Build time - The checker walked, for every reference to a symbol, all call statements that mention it. `all.push({..})` 20,000 times in one function takes 2.6% fewer instructions to build. No answer of the checker changes. `bun check` shares the file. - `bun_aot` is built at its own optimization level again, not at "s". The smaller Bun cost up to 0.85% more instructions a build. - The front end keeps a tree that it will be asked for again instead of parsing the file anew. The output is identical. The first two turn `build-scaling-6` green on Alpine x64. Following #44833 (`bun test`: Vitest compatibility) - An import with a query follows main's rule: `raw` among the `&` parameters is the file's text, `url` is the file loader. - Which file is the entry point of a test run is asked of the runner, not compared with the main module's path. - Tests that state what plain `bun test` does now state what it does on main: a name that a mock leaves out is `undefined`, not a SyntaxError; a factory that failed throws again at each load; `spyOn(namespace, "absent")` throws a TypeError. - `bun test --aot` doesn't match plain `bun test` on module mocks yet. Each such case says so behind a `TODO(bun)` and stays a comparison with plain, so it goes red the day the two agree: a name that only a mock has, a name that a mock leaves out, the order of two link errors where a file may be mocked, the default of a mock that has none, a load above or behind a mock whose factory failed. - `compile-trim-capabilities`: its reader of machine code took every `e8` and four bytes for a call, also in the middle of another instruction (`80 bb e8 96 00 00 01` is `cmpb $1, 0x96e8(%rbx)`). It now asks that an instruction begins there. - The test that compares the functions of a test run with those of the executable leaves out the map's "outlined" rows: `bun:test` now brings its fake timers into every test run. - Two tests of the engine's own suites are listed as failing: they count the entries of a table that has three more since #44833. Their fix is in the engine and comes with its next commit. Tests - `a-line-of-the-stage-log-goes-out-in-one-write` is a todo where the running glibc is 2.37 or newer. The engine's long log lines still go out in pieces there. The fix is in the engine and comes with its next commit.
…ended, faster loops (#971) * AOT: miscompilations and crashes found since #967, builds that never ended, faster loops Wrong answers of compiled programs: - A list of numbers with one `undefined` in it, iterated by two loops. The first adds the numbers up, the second counts the `undefined`s: it counted 0 instead of 1. - In a generator, a value read before a `yield*` and used after it was wiped when a parameter that held an object was set to `null` or `undefined` in the same loop: 12 instead of 22. A second program never ended. - A `RegExp` made from text out of a list was compiled from the text the build had predicted, even when the list was read under two names in one step and then changed. - `main.mjs` imports `m.mjs`, which imports `x.cjs`, which calls `require()` on `m.mjs` under a name made at run time: `main.mjs` ran after `x.cjs` had ended, not before the `require()` returned. - An `import()` made while other `import()` calls were waiting, and whose graph shares a module with one of them, was held back behind all of them. Crashes of compiled programs: - A segfault in eleven lines: a loop reads and stores a static field that holds an object, after the field was deleted while a Proxy was on the class's prototype chain and was defined again through the trap. - A segfault at address 0 after an out-of-memory error that the program catches: a rope of 2^30 two-byte characters that cannot be flattened, ahead of a short loop in a `try` with `catch` and `finally`. Builds that failed or never ended: - Thirteen lines did not build in two minutes: two nested loops of eight iterations, a swap in a `try`, an xor in its `finally`. B3's `Value::returnsBool()` recursed into both operands without remembering what it had asked, so its cost was exponential. - One function spent 214 of 246 seconds in type propagation: its queue was last in, first out. - The compiler aborted on `for (;;) { if (source.take === never) return ..; const got = source.take(..); .. }`. A loop that reads the same property of an unchanging object on every iteration gets a version that asks ahead of the loop, and a type test on that read was turned into a cast of the copy. Where both sides of the test's branch leave the loop, the branch still hands on what the test proved, so its condition has to stay a test. - Fourteen lines stopped the build with `InvalidGuard`: a read of a typed array at the top of an iteration relied on the bounds check at the end of the previous one, and the loop's counter was then replaced by a constant. - Quadratic time in large functions: three places in Air's register and stack allocators, two in the ahead-of-time compiler's own passes (one is flow analysis), and the lookup of a regular expression's symbols (arm64: twice as many alternatives cost 9.5 times as much, now 1.74 times). `bun test --aot`, following oven-sh/bun#44833: - An `import` that reaches a mock whose factory threw gets the error that the loader kept. `require()` of a mocked module returns what the factory returned. A later mock of a CommonJS file doesn't change what an earlier `require()` returned. - The engine's own tests no longer depend on how many built-ins Bun has. Speed: - New B3 pass, promoteLoopMemory: `a[k] += v[j]` and `o.total += x` reloaded on every iteration what the previous one had stored. The word is read once ahead of the loop and carried in a Phi. The stores stay where they are. Off by default; only the ahead-of-time compiler's policy turns it on. - `map.has(k)` followed by `map.get(k)` searches once. Keys that are objects are searched inline in loops. - A generator function made for each call: 2,585 to 175 instructions. tslib's `__awaiter`: 3,876 to 1,441 per iteration. - Helpers that Babel and TypeScript emit: `Reflect.construct` behind tests, a built-in bound and called in one function, descriptors read and defined again, `for..in` copies. - Remainders and quotients of whole numbers stay integers. A loop polls with one faulting load on arm64, macOS x64 and Windows arm64 too. * AOT: a library function that a test run leaves out is refused, not aborted on `import { test, vi } from "vitest"; test("t", () => { vi.useFakeTimers(); })` under `bun test --aot` ended in `abort()`. The code that throws "was not compiled into this executable" looked the function up in the table of built-ins to ask whether it is an accessor's getter, and a function that the table leaves out has no record there. It asks `contains()` first now, so the error is thrown. * AOT: a member copied off `require("bun")` under a computed key is followed to its call `bun = require("bun"); copy = {}; copy[key] = bun[key]; copy.plugin(options)`: the build now sees that `copy.plugin` may be `Bun.plugin`. The test, `plugin-copy-late-mark.ir`, comes with Bun's half. * AOT: two segfaults: a store through a caught error's `constructor.prototype`, and to a method found on `Bun.s3` `TypeError.prototype.list = [1, 2]`, then `catch (e) { put(e.constructor.prototype, { length: 3 }) }` with `put(where, what) { where.list = what }`, then `TypeError.prototype.list.length` read in a function: the read took the list for what the build had seen stored by name, and the program ended in a segfault. The same with `Bun.S3Client.prototype.file.list = [1, 2]; Bun.s3.file.list = { length: 3 }`. Where no list was involved, the read answered `undefined`. Both stores are now seen by a read that names the object. * AOT: promoteLoopMemory is not switched on yet The pass and its tests stay in B3. The ahead-of-time compiler's policy no longer turns it on. What broke, in a loop: holder.n = (holder.n + 1) | 0; if (j === when) other.n = 50; if (holder instanceof Other) bump(holder); with `function bump(o: Other) { o.n = (o.n + o.k) | 0 }`. Where `holder` and `other` are one object, a `Plain`, the store of 50 was lost: the function answered 7:29 instead of 55:317. The copy of `bump` in the loop never runs for that object, but its access named a narrower heap for the word, and the pass let that one heap decide that nobody else writes it.
…s, a bounds test on split() (#972) * AOT: miscompilations and crashes found since #967, builds that never ended, faster loops Wrong answers of compiled programs: - A list of numbers with one `undefined` in it, iterated by two loops. The first adds the numbers up, the second counts the `undefined`s: it counted 0 instead of 1. - In a generator, a value read before a `yield*` and used after it was wiped when a parameter that held an object was set to `null` or `undefined` in the same loop: 12 instead of 22. A second program never ended. - A `RegExp` made from text out of a list was compiled from the text the build had predicted, even when the list was read under two names in one step and then changed. - `main.mjs` imports `m.mjs`, which imports `x.cjs`, which calls `require()` on `m.mjs` under a name made at run time: `main.mjs` ran after `x.cjs` had ended, not before the `require()` returned. - An `import()` made while other `import()` calls were waiting, and whose graph shares a module with one of them, was held back behind all of them. Crashes of compiled programs: - A segfault in eleven lines: a loop reads and stores a static field that holds an object, after the field was deleted while a Proxy was on the class's prototype chain and was defined again through the trap. - A segfault at address 0 after an out-of-memory error that the program catches: a rope of 2^30 two-byte characters that cannot be flattened, ahead of a short loop in a `try` with `catch` and `finally`. Builds that failed or never ended: - Thirteen lines did not build in two minutes: two nested loops of eight iterations, a swap in a `try`, an xor in its `finally`. B3's `Value::returnsBool()` recursed into both operands without remembering what it had asked, so its cost was exponential. - One function spent 214 of 246 seconds in type propagation: its queue was last in, first out. - The compiler aborted on `for (;;) { if (source.take === never) return ..; const got = source.take(..); .. }`. A loop that reads the same property of an unchanging object on every iteration gets a version that asks ahead of the loop, and a type test on that read was turned into a cast of the copy. Where both sides of the test's branch leave the loop, the branch still hands on what the test proved, so its condition has to stay a test. - Fourteen lines stopped the build with `InvalidGuard`: a read of a typed array at the top of an iteration relied on the bounds check at the end of the previous one, and the loop's counter was then replaced by a constant. - Quadratic time in large functions: three places in Air's register and stack allocators, two in the ahead-of-time compiler's own passes (one is flow analysis), and the lookup of a regular expression's symbols (arm64: twice as many alternatives cost 9.5 times as much, now 1.74 times). `bun test --aot`, following oven-sh/bun#44833: - An `import` that reaches a mock whose factory threw gets the error that the loader kept. `require()` of a mocked module returns what the factory returned. A later mock of a CommonJS file doesn't change what an earlier `require()` returned. - The engine's own tests no longer depend on how many built-ins Bun has. Speed: - New B3 pass, promoteLoopMemory: `a[k] += v[j]` and `o.total += x` reloaded on every iteration what the previous one had stored. The word is read once ahead of the loop and carried in a Phi. The stores stay where they are. Off by default; only the ahead-of-time compiler's policy turns it on. - `map.has(k)` followed by `map.get(k)` searches once. Keys that are objects are searched inline in loops. - A generator function made for each call: 2,585 to 175 instructions. tslib's `__awaiter`: 3,876 to 1,441 per iteration. - Helpers that Babel and TypeScript emit: `Reflect.construct` behind tests, a built-in bound and called in one function, descriptors read and defined again, `for..in` copies. - Remainders and quotients of whole numbers stay integers. A loop polls with one faulting load on arm64, macOS x64 and Windows arm64 too. * AOT: a library function that a test run leaves out is refused, not aborted on `import { test, vi } from "vitest"; test("t", () => { vi.useFakeTimers(); })` under `bun test --aot` ended in `abort()`. The code that throws "was not compiled into this executable" looked the function up in the table of built-ins to ask whether it is an accessor's getter, and a function that the table leaves out has no record there. It asks `contains()` first now, so the error is thrown. * AOT: a member copied off `require("bun")` under a computed key is followed to its call `bun = require("bun"); copy = {}; copy[key] = bun[key]; copy.plugin(options)`: the build now sees that `copy.plugin` may be `Bun.plugin`. The test, `plugin-copy-late-mark.ir`, comes with Bun's half. * AOT: two segfaults: a store through a caught error's `constructor.prototype`, and to a method found on `Bun.s3` `TypeError.prototype.list = [1, 2]`, then `catch (e) { put(e.constructor.prototype, { length: 3 }) }` with `put(where, what) { where.list = what }`, then `TypeError.prototype.list.length` read in a function: the read took the list for what the build had seen stored by name, and the program ended in a segfault. The same with `Bun.S3Client.prototype.file.list = [1, 2]; Bun.s3.file.list = { length: 3 }`. Where no list was involved, the read answered `undefined`. Both stores are now seen by a read that names the object. * AOT: promoteLoopMemory is not switched on yet The pass and its tests stay in B3. The ahead-of-time compiler's policy no longer turns it on. What broke, in a loop: holder.n = (holder.n + 1) | 0; if (j === when) other.n = 50; if (holder instanceof Other) bump(holder); with `function bump(o: Other) { o.n = (o.n + o.k) | 0 }`. Where `holder` and `other` are one object, a `Plain`, the store of 50 was lost: the function answered 7:29 instead of 55:317. The copy of `bump` in the loop never runs for that object, but its access named a narrower heap for the word, and the pass let that one heap decide that nobody else writes it. * AOT: fix crashes and miscompilations found since #971, free Map/Set index slots, turn promoteLoopMemory back on Stacked on #971. The stored programs and tests for everything listed here come with oven-sh/bun#47605. Crashes and failed builds: - `TypeError.list = [1, 2]; JSON.parse("0", () => Reflect.set)(e.constructor, "list", { length: 3 })` with `e` a caught error, then `TypeError.list.length` in a function: the compiled program segfaulted. The same happened with a hidden `Object.assign` or `Object.defineProperty`, and when the writer was called as a member of a hidden object. A write through a value the compiler can't identify is now also applied to built-in objects, even when flow analysis sees no path to them. - `Object.defineProperty(target, key, Object.getOwnPropertyDescriptor(source, key))` with a `key` object whose `toString()` returns "a" and then "b": plain prints 3, compiled code segfaulted. With methods the wrong one ran; with getters it threw "'a' was left out of the executable". - Every `--aot` executable aborted before its first statement on a kernel with 64 KB pages (RHEL 8 aarch64, Oracle's UEK, the 64k kernels for NVIDIA Grace). - The compiler aborted with "Stopped while compiling run in ConvertCountersToIntegers" on `for (..) { if (j === w) other.n = j; holder.n = (holder.n + 1) | 0; if (holder instanceof Q) bump(holder); }`. - `vi.doUnmock("./k.js")` while a plugin's async `onLoad` is still loading another import segfaulted under `bun test --aot`. - `bun test --aot` aborted under `validateExceptionChecks`, which CI's ASAN lane sets, when a mock was installed in a layered run: `vi.doMock("./m08.js", async () => { await sleep(80); return { value: "mocked" }; }); import("./m08.js")`. Miscompilations: - The code that creates, fills and stores a module variable's value is deferred until the variable is first read. An assignment to a plain object or function created by that initializer was deferred with it. But a program may call `Object.freeze(Object.prototype)` or `Object.defineProperty(Object.prototype, "toString", { writable: false })`, and after that an assignment to an inherited name fails: a TypeError in strict code, silently dropped in sloppy code. Imports run first and the main module hardens afterwards, so plain did the store before the freeze and compiled code did it after. main has this bug too. - A loop's stores to a variable were delayed to the loop's exit edges, even across an operation that throws only when memory runs out (`text.charCodeAt()` on a rope). That error can be caught, and then the loop is left through none of those edges. - A Map or Set lookup with the same key on every iteration was hoisted out of the loop, even when a `try` covered the lookup but not the loop's preheader, and even when the loop might not run: `for (let i = 0; i < 0; i++) map.has(rope)` threw a RangeError. - A loop that does `const list = []`, pushes, reads `this.state.type`, reads `list[0]` and stores the list into a node at the end of the iteration: a value read back out of an open object was taken for this iteration's list. Also with the push on one arm of an `if`. - `seen.push([a])` in one loop and `for (const kept of seen) show(kept)` in the next: the inner list was attributed to `show()`. - `const o = Object.create(Blob.prototype, {}); o.constructor = { get name() { return "Stored" } }; o.text()`: plain throws ERR_INVALID_THIS ".. received an instance of Stored"; compiled code threw "'name' was left out of the executable". The same for `Buffer.concat(v)` when `v` is an object that isn't an array. - `const { module } = await WebAssembly.instantiate(..)`: the compiler didn't know what the promise resolves with. Modules: - `for (3 times) try { require("./t.cjs") } catch {}` where t.cjs throws: the body ran once and the first error was thrown three times. It now runs again each time, as in node and `bun run`. - `Object.isExtensible(require("./e3.mjs"))` after an import of the same file: plain true, compiled false. - `require("./m3.mjs"); Object.keys(require("./d.json"))` where m3.mjs imports d.json: plain `a,default`, compiled `a`. - Loading a file that only a plugin can provide, under a name computed at run time. - A test registers `Bun.plugin` with an `onLoad` that matches slow.js and then does `await import("./user.js")`, where user.js imports k.js and slow.js: "Cannot access 'x' before initialization" for k.js's export. Quadratic behavior: - `map.delete(k); map.set(k, v)` took a new slot in the table's index each time and never freed the old one, so each iteration cost the size of the table. A deletion now frees the slot when no search can miss it. This costs 5 more instructions on a set and on a delete. - `class Below extends mix(Base)`: the whole caller was walked for every such call. At 3,200 classes that step went from 8,306.6 M instructions to 119.7 M; the generated code is byte-identical. - Two more build steps that grew with the square of the program: choosing representations around a web of values, and finding who reads a parameter that is only passed on. Speed: - The AOT policy turns promoteLoopMemory on again, with the fix for the aliased store that got it turned off in #971. What guards it: 14 stored programs that each gave a wrong answer with an earlier version of the pass, a fuzzer family written for it, 8,815 programs with no differing result, and a review by someone who didn't write it and tried to break it: 0 of 7,728 programs wrong. - The low word of a sum that was built from doubles is computed as a 32-bit sum. Outside aot/: 13 files. The JIT compiles b3/ too; the changes below run there only where noted. - b3/B3PromoteLoopMemory.cpp and its testb3 cases: the pass. It runs only when `loops.promotesMemory` is set: false by default, set only by the AOT policy. - b3/B3LikelyPaths.{h,cpp}: how the nearest common dominator over likely paths is found. Used by hoistLoopInvariantValues() under `loops.hoistsOnlyFromLikelyPaths`: false by default, set only by the AOT policy. - b3/B3ReduceStrength.cpp and testb3_bun_reductions.cpp: `Trunc(Op(a, b))` becomes `Op(Trunc(a), Trunc(b))` when this walk built the operation from doubles. Under `narrowsIntegers`: false by default, set only by the AOT policy. - b3/B3LoopRangeChecks.cpp, b3/B3Policy.h: a switch that lets a test sabotage the compiler (`omitsLimitOfClampedBound`, false by default) and a client callback (null by default). Loops are versioned only under `loops.versions`: set only by the AOT policy. - b3/B3CompilationReport.h: 16 more counters in a Procedure's report. That is 64 more bytes per Procedure, in the JIT too; nothing there increments them. - heap/HeapSubspaceTypes.h: an #include under the existing `#if ENABLE(AOT)`, for the generators' own subspace. - llint/LLIntEntrypoint.cpp, runtime/VM.cpp: inside `#if ENABLE(AOT)`: code that runs in the interpreter is also counted in a process without an image, under an option that tests set. - runtime/ExceptionHelpers.cpp: inside `#if ENABLE(AOT)`: a clearer error for a global function declaration that reuses the name of a global that compiled code treats as a constant. * AOT: a bounds test on the result of split() or splice() compared against the wrong length function f(h: string | null): string[] { if (h == null) return []; const accepted: { type: string; count: number }[] = []; for (const part of h.split(",")) { const [type, ...params] = part.split(";").map(v => v.trim()); accepted.push({ type, count: params.length }); } return accepted.map(e => e.type); } Built with --aot-validate, f("a/b") and f("c/d, e/f") returned [] instead of ["a/b"] and ["c/d", "e/f"]: the whole for-of loop was removed. No error, exit code 0. main has this bug too. When only the first few elements and the length of an array returned by split, match, exec or splice are used, allocation sinking never builds the array. It becomes a virtual object whose shape counts only the elements that are kept separately. evaluateBoundsTest() compared an index against that count as if it were the array's length. Nothing in this function reads a constant index, so the count was 0, every index was "out of bounds", the edge into the loop body was dead, and the body held the only use that would have asked for the real array. For such an array, an index past the shape's count is now unknown instead of false. The wrong answer has only been seen under --aot-validate, where a type check keeps the array alive. At default flags, iterating over split()'s result becomes a loop over indexOf and substring and no array is made. But the rule itself was reached at default flags too, so a wrong answer there can't be ruled out for earlier builds. The same shape with splice() was wrong as well. The generated code of ten servers and the TypeScript compiler benchmark is unchanged at default flags. The tests come with oven-sh/bun#47605: test/aot/parallel/strings/split-list-walked.ts and test/aot/parallel/arrays/spliced-list-walked.ts. * AOT: include what AOTValueFlowSolveTestCommands.cpp uses The file uses StringPrintStream and makeString() without including their headers. It compiled only where the unified build put it next to a file that does. The -aot lanes failed with "unknown type name 'StringPrintStream'" and "use of undeclared identifier 'makeString'". * AOT: don't name a variable `small` in AOTValueFlowCoreTests.cpp The Windows SDK's rpcndr.h has `#define small char`, so `ValueSets small(1u << 10);` didn't compile on the Windows -aot lanes ("cannot combine with previous 'type-name' declaration specifier"). * AOT: a module whose mock ended before its body ran has not begun to load Under `bun test --aot`, two test files each mock the same ES module before it loads (`vi.mock("./dep.js", ..)` hoisted, or `mock.module()` in a test ahead of the `import()`). A third file, which mocks nothing, read the second file's mock, by name and through the namespace object, and dep.js never ran. Plain `bun test` runs dep.js and reads the real values. When the first mock ended, its module still counted as having begun to load, because that goes by the group it was loaded with, which is never taken back. So the second mock's factory didn't wait for the first import. It ran at once, and its values were laid by name over a module whose body had never run. Modules::hasBegunToLoad() now answers no for such a module. The bit it reads is set only where a mock that stood in place of a module ends, so an executable never gets here. The test comes with oven-sh/bun#47605: test/aot/mock-second-file.test.ts. * AOT: check for an exception after a mock in place is taken back AOT::didEndMock() had no throw scope, and went on after setMockInPlace(null), which can throw. Under `validateExceptionChecks`, which CI's ASAN lane sets, `bun test --aot` aborted in two test files: "setMockInPlace @ AOTModules.cpp .. unchecked as of captureExports @ AOTModuleLinking.cpp". // a.test.js vi.mock("./dep.js", () => ({ value: "mock" })); // the mock ends with its test file; b.test.js then imports ./dep.js * AOT: build the same program at one thread as at many The `typescript` package built at `--aot-threads=1` and at 4 gave two different executables from the same front-end output. One line in one function of 16,981 differed: for one property read, the predicted receivers were empty at one thread and listed at two or more. No proof differed, and 4, 4 again and 16 threads agreed, so it was never run to run. findOnAllThreadsAndDo(), in the pass that computes predictions after the solver's rounds, was the only place in the flow analysis with a test for exactly one thread. With one worker each item of a wave ran in place and saw what the items before it had done. With more, all steps of a batch are computed from the state at its start and replayed in order. Both end at a fixpoint, but the pass has caps, so not always at the same one. One thread now takes the same path as many. Batch and share sizes don't depend on the thread count. At two or more threads nothing changes. Six thread counts from 1 to 32, eight times each: one hash, the one that 4 and 16 threads gave before. test/aot/same-program-however-the-threads-fall.test.ts in oven-sh/bun covers it. * AOT: a read copied for another kind of array storage kept its proof across a store through a second name function clean(src, dst, n) { for (let i = 0; i < n; i++) { if (src[i] < 0) dst[i] = NaN; // a bad sample else dst[i] = src[i] * 2; } } clean("1,2,3,4,5,6".split(","), [], 6); const samples = [1.5, -1.5, 2.5, 3.5, 4.5, 6.5]; clean(samples, samples, 6); console.log(samples.join(" ")); // 3 NaN 5.5 7 9 13, should be 3 NaN 5 7 9 13 Silent, at default flags, in plain JavaScript and in TypeScript. main has this bug. PlanArrayCapacities copies the reads behind a test of an array's kind of storage for the other kind, as far as the test's proof goes. addReadsOfOtherStorage() copied the stores between those reads with them and never asked whether the copy's proof outlives such a store. A store through another operand that may be the same array ends it. The guards are now asked again behind each copy, and a copy that may lose its proof is taken back. New: --aot-sabotage=CopiedReadsKeepTheirProof, the statistics ReadsCopiedForOtherStorage and CopiesOfReadsTakenBack, two IR tests. The 30 stored programs come with oven-sh/bun#47605. * AOT: SplitBehindEntryChecks kept value ids across a retyping that renumbers the values function outer(p: any, parent: number | null): string { const head = parent === null ? "" : "<a " + parent + ">" + "<b " + parent + ">"; return head + (p as Kind).name; } // called hot as outer(given, null) An assertion build of the compiler aborted on `id.index() < values.size()`. A release build read outside the function's table of values. No release build has been seen to give other code. typeValuesFromNever() may cut dead blocks, and then it compacts the function and gives every value a new id. sayWhatOperandsOfFormerChecksHaveProven() went on with the ids it had taken before. The retyping is now handed a Renumbering: the checks' ids are mapped, and those cut with the dead code are dropped. The bound at the head of that function is a RELEASE_ASSERT. * AOT: the second import() of a CommonJS file whose mock factory throws asks the factory again mock.module("./lib.cjs", () => { throw new Error("boom " + ++n); }); // import(), require(), import(), require(), each caught Under `bun test --aot` as one program the errors read boom 1, 2, 1, 3. Plain `bun test` has 1, 2, 3, 4. An import() of a CommonJS file begins at the file's ES face. The walk then comes to the file itself, which isn't the root, so the throw counted as "behind an import statement" and the error was kept on the file for the loader. The file behind the root's face now counts as the root. * AOT: a mock factory that takes importOriginal and fails is asked again at the next import In plain `bun test` a factory that takes `importOriginal`, sync or async, is laid over the original. Where it fails the loader keeps no record: the next import asks it again, and the module that imports the mocked one then loads. A factory without the parameter keeps its error. Compiled code kept the error for both kinds. deferMock() is now told which kind the mock is, and runMocksBelow() keeps no error for the first kind. Where such a factory's promise rejects a turn later, the mock waits for the next load again instead of being taken away. The test comes with oven-sh/bun#47605: test/aot/mock-factory-retry.test.ts. * AOT: vi.doUnmock() in a test, behind a hoisted vi.mock() of a module that never ran vi.mock("./dep.js", () => ({ value: "mock", fn() { return "mock-fn"; } })); import { value, fn } from "./dep.js"; test("t", () => { vi.doUnmock("./dep.js"); fn(); value; }); Under `bun test --aot` the imported function answered `real-fn` silently, and consts, `ns.value` and `Object.keys(ns)` threw "before initialization". In plain `bun test` whoever was linked keeps the mock. An ended mock that stood in place of a module is now let go when a load next comes to that module. At the end of the test file, while a graph is being loaded, and for .cjs and library modules it is let go at once, as before. So it is where a module that a require() has already answered imports the mocked module: a compiled require() of a module that has run asks nobody, so no load would ever come by. * AOT: `export * from "<a CommonJS package>"` in a mocked module, in layered test runs // lib.mjs export * from "driver"; export let v = 1; // a.test.js mock.module("./lib.mjs", () => ({ v: "mock v" })); await import("./lib.mjs"); In a layered `bun test --aot` run (`--parallel=2 --aot-memory-limit=1KB` is enough) this failed with `TypeError: The module ".../driver/index.js" is not loaded`. The same error came in a second test file that only imports a user of a module which the first file had mocked. A regression of this branch. Three places went behind the `export *` of a mocked module to take names, and evaluateModulesOwedBelow() ran a module that a mock had kept from running without asking the loader for what only that module imports. A mock that came ahead of the load now stands for the whole module. * AOT: the first require() of an ES module gave its namespace object a Structure shared with others import { x } from "./a.mjs"; // read hot const all = ["./a.mjs", "./b.mjs", "./c.mjs"].map(p => import.meta.require(p)); function readX(ns) { return ns.x; } // over the three objects, hot In a layered `bun test --aot` run readX() came to answer a's x for b. Silent. A regression of this branch, found in review. willRequireESModule() set the Structure that all extensible namespace objects share, also on an object that had a Structure of its own, and hasOwnStructure() went on answering true. A property cache keyed on the shared Structure then carried the first module's binding to the next. Such an object now gets a new extensible Structure of its own. * AOT: the compiler aborted on `o.self = o` where o may be a subclass of Map class C extends Map {} function see(o) { return o.self; } function run(o = new C()) { see(o); o.self = o; } run({}); run(); "Stopped while compiling run in PlaceTypeTests." Also over Set, WeakMap, WeakSet, Array and Error. main has this bug. Behind a store's tests, rewriteTestedPropertyAccess() takes the last TypeTest for the test of the stored value, which says nothing about the receiver, and leaves it out. Here the stored value is the receiver, so its only test was left out and the access couldn't be resolved. The test is now left out only where the receiver hasn't passed it too. * AOT: an import() waits for the async factory of a mock below it vi.mock("./a", async importOriginal => ({ ...(await importOriginal()), a: "mocked-a" })); import { a } from "./a"; // and user.ts: export const atLoad = a; Under `bun test --aot` as one program: "Cannot access 'a' before initialization", or the real value, and the next test file "never finished loading". Which of the three came went by when a thread had read the original from disk. A hoisted vi.mock() turns the test file's import statements into such import() calls, so Vitest's commonest partial mock met it. main has this bug. startImport() called the waiting mock's factory, held the import for one turn and delivered it whatever the factory had done by then. Only the loader's own fetch waited. Where factories wait, the import now goes on behind them, and is rejected with what one of them throws. * AOT: a bounds test on the kept prefix of split()'s result compares against the real length Follows "a bounds test on the result of split() or splice() compared against the wrong length". That commit made an index past the kept elements unknown instead of false. The other side had the same flaw: an index inside the shape's count was taken for in bounds, though the array may be shorter than the number of elements that the function reads. The test becomes `position < the array's real length`. New: --aot-sabotage=ListsOfBuiltinsEndWithWhatIsKept and the statistics ListsOfBuiltinsNeverBuilt and ObjectsOfBuiltinsNeverBuilt. It costs 0.15% on one router idiom. * AOT: three IR tests check the new counters of arrays and objects that are never built * AOT: a ring of `export *` with a package in it hung the link of a layered test run // index.ts: export * from "./a"; export * from "./b"; export * from "pkg"; // a.ts, b.ts: export * from "./index"; import { missing } from "./index"; In a layered `bun test --aot` run the link never ended (485,000 frames when it was killed). It should throw the SyntaxError for the missing export, as it does as one program and in an executable. A regression of this branch, found in review. Modules::isMockedInItsImage() called itself for every dynamic `export *` with no set of visited modules. It is one loop with a BitVector now, and asks each module once. * AOT: a name that a barrel hands on from a mocked CommonJS file is the mock's // barrel.js: export { name } from "./lib.cjs"; vi.mock("./lib.cjs", () => ({ name: "mock" })); import { name } from "./barrel.js"; Under `bun test --aot` as one program `name`, `ns.name`, Reflect.get and `(await import("./barrel.js")).name` read `undefined`. Object.getOwnPropertyDescriptor and a spread of the same object were right. findNamespaceExport() leaves a member to the namespace object where the module that declares the variable may be mocked. The names of a CommonJS file are variables of its ES face, and the mark sits on the file, so the read was bound to a variable that nothing fills while the mock stands. Both are asked now. * AOT: a function of a built-in module replaced by a writer the build can't see import found from "node:module"; JSON.parse("0", () => Reflect.set)(found, "SourceMap", mine); function use() { return found.SourceMap(); } use(); // plain: mine's result; compiled: SIGSEGV With `buffer.atob`, `Buffer`, `Blob` and all of `node:util/types` the write went through, a read saw it, and the call by the module's name silently reached the original. main has this bug. Flow analysis reasoned that start-up locks every member of a built-in that no statement writes. Nobody locks the object of a module, and nobody should: stubs and instrumentation replace such members. A holder that is a module's object, has no lock under the name and has escaped to code nobody sees now also answers what was stored through the unknown object, so the call is tested and not bound. * AOT: a variable read stale behind a method picked by a name known only at run time const handlers = { open(done) { return { run() { done({ ok: true }); } }; }, close(done) { return { run() { done({ ok: false }); } }; }, }; const type = JSON.parse('{"type":"open"}').type; let result = null; handlers[type](r => { result = r; }).run(); console.log(JSON.stringify(result)); // null, should be {"ok":true} Silent, at default flags. Neither the verifier nor --aot-validate sees it; --aot-check=MovedLoads does. main has this bug. In cold code Select fuses a property read and a call into CallMethod. That operation's effects are "unknown code", without its known callees. "Unknown code" is what functions entered by unknown code do, and a function whose callers flow analysis all knows isn't among those. So B3 was told that the call doesn't write the variable's word, and forwarded the earlier store to the load. The read and the call are now fused only where the call's effects lie within those of unknown code. New: --aot-sabotage=FusedCallsEnterOnlyUnknownCode. * AOT: a setter laid by Object.defineProperty that stores through `this` import "node:util"; class C { z = 0; } function run(given) { const o = given ?? new C(); Object.defineProperty(o, "p", { set(v) { this.z = v; } }); o.p = 1; return o.z; } console.log(run(), run([])); // 0 1, should be 1 1 Silent. Where a TypeScript class declares `z?: number` the executable aborted. Twenty built-in modules arm it by being imported anywhere in the program. main has this bug. The library's text holds a function that lets accessors escape, so the setter escapes, rightly. For a store with several receivers the setters found are then {unknown}, which covers the known setter, and the branch that hands each receiver to the setters on its own chain asked the covered view and was skipped. The instance was neither handed to the setter nor made to escape, so its field counted as stored once. For a store, where unknown stands among the setters found, the receivers that haven't escaped are now told apart, and each is handed to the setters laid on its own chain. A getter that writes through `this` and is run by a spread had the same hole in the rule for copies. What flow analysis hands the optimizer is unchanged on express, fastify and the typescript package. * AOT: the test of a branch that passes on a proof stays in the branch's block function f(x, n) { let fuel = 3, b = x, c = 1n, e = x, a = x; const gt = { get v() { a = b; return 1; } }; for (let i = 0; i < n; i++) c = e; while (c === b && fuel-- > 0) { gt.v; } b++; return e + 1; } f(1, 2); f("a", 2); "Stopped while compiling f in SimplifyControlFlow", and the step that the message advises to switch off isn't the one at fault. main has this bug; a neighbour with a value out of Map.get built on main and aborted on this branch. HoistLoopInvariantReads lifted the loop's test out of the block of its Branch, which passes a proof on its taken edge. PlaceTypeTests then copied a part of the loop, and the SSA repair turned the Branch's condition into a block parameter, which proves nothing. Of 58 varied forms 23 didn't build and none built and answered wrongly. Such a test is no longer hoisted. Those loops now make an unchanging test in every iteration: 1,031 functions of the public corpus change, 3,361 bytes fewer. * AOT: a class is no longer recognized by its prototype's own `constructor` property const Base = t => class { constructor() { this.a = t; } }; const Heir = S => class extends S { constructor() { super(); this.b = 5; } }; for (let i = 0; i < 2; i++) { const D = Heir(Base(i)); delete D.prototype.constructor; console.log(i, JSON.stringify(new D())); // the second iteration: SIGSEGV } With class fields it is silent: `{"a":1}` for `{"a":1,"b":5}`. Besides `delete`: Reflect.deleteProperty, an accessor, Object.defineProperties, and a mixin helper that copies another prototype's own properties. Of 3,432 generated forms 472 were wrong, 164 by SIGSEGV. main has this bug. From a class's second evaluation on, where no member guard is left, its class object and prototype object are left out of the table of holders. Asked later whose prototype an object is, ClassTable::classOfHolderLeftOut() read the object's own `constructor`. With that gone the answer was "nobody's", and the base's constructor kept the object it had allocated in the base's shape. A prototype object that is left out now names its class object under a private name of the class table: read-only, not enumerable, not deletable, and seen by nothing that a program can ask. * AOT: include what AOTCallRules.cpp uses matchMethodCall() now asks context.build().isSabotaged(..), and BuildContext was an incomplete type there. Bun's build precompiles a prefix header that brings AOTBuildContext.h in; WebKit's own CMake build does not.
#967, #971 and #972 changed how an import() waits for its text. Two files of aot/runtime had conflicts. - TextThatWaits has the new fields of main and then the async context of the call. - deliverFirstTextThatWaits() installs that async context around the call of startImport() that main makes. - The jobs that main queues in startImport() and decideHowCountedImportGoesOn() still carry the async context of the import().
main has eight commits since 4d45755, the commit this branch started at: #967, #971, #972, #974, #978, #979, #982 and #983. Two files conflict, each in one place. Both sides changed a line that calls one of WTF's atomic functions: this branch gave it the MemoryOrder template argument (WebKit/WebKit@6ab0aebc81), main changed the code beside it. - aot/analysis/AOTCallGraph.cpp: main's new loop over body.expectedFunctionTypes, with atomicLoad<MemoryOrder::Relaxed>() and atomicStore<MemoryOrder::Relaxed>(). - aot/runtime/AOTInstance.cpp: main's `if constexpr (hasFaultingLoopPolls)` in setPollsEnabled(), behind atomicStore<MemoryOrder::Relaxed>(). No conflict, and the same change: aot/valueflow/AOTFlowApply.cpp has three calls in the old form that main added. They take the template argument. The other 21 files that both sides changed merge as they are.
The Bun side of the rewritten ahead-of-time compiler. The engine side is oven-sh/WebKit#997, which has the design and the numbers, with its follow-ups oven-sh/WebKit#999 and oven-sh/WebKit#1000. This is #47942 with its follow-ups #48088 and #48112 in it. `src/aot`: the lowering writes the program that the engine's new scan, link and walk read. Captured values are copied into closures where nothing stores to them later, straight-line blocks are joined, and CommonJS and ES module graphs are resolved at build time, cycles included. `bun test --aot`: module mocks, spies, hoisted factories, re-evaluated modules and `--rerun-each` behave as they do without `--aot` in many more cases. Type errors stop an `--aot` build, in TypeScript only; `bun test --aot` prints them as warnings and runs. A computed import no longer embeds data files that its written name does not ask for. `bun:ffi`: no limit on the number of open callbacks in compiled programs. What changes for Bun without `--aot`, by folder and file, is listed in the description of the pull request; the list below is the short form that the commits carried. In these lists "old" says that an earlier commit of the branch had it too and "new" that the last commits brought it; neither says anything of the compiler that main had before. "As users build" is `bun build --compile --aot`; a stored program or test run is a test that this commit adds. The commits of the three branches, each with what it fixes and what was run for it, stay readable on the pull requests. Fixed on the branch since the description of the pull request was written: - In the engine, each with its JavaScript in the messages of oven-sh/WebKit#999: silent, as users build: `receiver.f(swap())` called the function that the argument had just stored; a later source of `Object.assign()` or of a spread was read as it was made, not as it is by then, and `Object.assign(t, s, t)` laid the old fields over the new; the order of an object's keys could be wrong behind a store during which code of the program runs. Silent, test runs with module mocks: the original of a mocked barrel had no names where two stars lead to one CommonJS file; the factory of a mock that requires its own CommonJS file got a TypeError. A build could hang on a program with a very wide switch, object or module. A build could take much memory and time on a program that makes many long lists as it starts. Right typed programs were refused with TS100031 (an `int32` out of `slice()` or `filter()`). Slower, no answer wrong: executables built with `--external` or `--aot-allow-code-from-disk` lost sight of what the library's internal files hand out. Stored programs, test runs and test files for them come with this commit. - Old, silent, in executables alone: `module.exports = function () { return [typeof this.children, typeof this.loaded].join(); }; console.log(module.exports());` printed `undefined,boolean` where `bun <file>` prints `object,boolean`; `this.children.length` threw. What `module.exports` is called with is the module object, and it goes where the callee takes it; so does a name that may hold the module object (src/aot/builtins/resolve/module_objects.rs, its tests). 6 stored programs and IR tests. - Old, silent, low: an error that `bun <file>` throws was not thrown: `console.log("start"); lib.f; var lib = require("./x.mjs"); console.log("end", lib.f());` printed `start`, `end 1` where `bun <file>` prints `start` and throws a TypeError: the read was taken out. A member of what `require()` gives may go only where the variable holds it already (src/aot/compile/removable.rs). 12 stored programs. - Old, silent or loud: one data file loaded under several types, one of them under a name that is made at run time: every such site, and every statement behind it, got the form of the site that is written first, where `bun <file>` goes by time: with yaml written second and run first, `ENOENT reading`, then the string, for `object,object`. A made name takes no part in the forms that the build shares (src/sema/program/code_walk.rs, code_load.rs, code_imports.rs). 2 stored projects. It is a part of the kind that stands under "Known problems", not all of it. - Old, an abort as users build: `process.finalization was left out of the executable, because nothing seemed to use it`, exit 134, where `dispatchEvent` is read under a computed key and called without a receiver and the name stands only in a template or in JSX text. The net that decides which later files are asked reads both (src/aot/builtins/resolve/unnamed_uses.rs). 11 stored programs. Four spellings are fixed, others of the kind are not. - Old, loud: a right typed program was refused with TS100031: a whole number that a type lists (`1 | 2 | 3`), read off an object and reckoned with where the read stands. The type is stated where the read is reckoned with; it proves nothing: each guess on it is tested at the run (src/aot/lower/convert/edges.rs). 1 stored program. Not fixed here: the same read into a local that is reckoned with. - Old, loud, Bun's half: `const load = [require][JSON.parse("0")]; Object.keys({ "node:zlib": 1, "bun:jsc": 1 }).map(load)` threw MODULE_NOT_FOUND where `bun <file>` and a plain executable answer. Where a load goes by a name that nobody follows, the executable holds the library modules whose names the program writes; a key of a literal, of a store, of a class member, a joined key and a key of a data file are heard as a text of that name is, if they are more than one word (src/aot/load/contents/load_names.rs, scan.rs, tests.rs). 3 stored programs. - In the engine, in the messages of oven-sh/WebKit#999. Old: a test run that mocks a module and ends the mock could crash. - docs/compiler/compatibility.mdx: 24 places now say what the compiler does: differences from `bun <file>` that are known and not fixed here are written down as differences. - In the engine, with its JavaScript in the messages of oven-sh/WebKit#999. Old, silent, as users build: a field into which the program is only seen to store objects that it fills under computed keys; code made from a text stores a list there, and a loop of nine turns asks `Array.isArray(shelf.bins)`: false, where `bun <file>` says true. So too where a listener stores its argument there: `const lay = bins => { shelf.bins = bins; }; new EventEmitter().on("filled", lay).emit("filled", [1, 2, 3]);`. 6 stored programs. - In the engine. Where the factory of a mock requires its own CommonJS file and that file throws on its way, the file runs again at its next `require()`, as under `bun test`; before, the answer was a TypeError. 9 files of stored test runs. - Old; silent where the sources still stand, loud elsewhere: `await import(flag ? "./other.sqlite" : "./data.sqlite", { with: { type: "sqlite" } })`, and the same under a name with a hole. An executable of `--compile` opens the database of the folder in which it runs; with --aot it opened the database of the folder in which it was built, under its whole path; where that is gone, "SQLiteError: unable to open database file". Beside it: a name that the build cannot see (`import(JSON.parse('"./data.sqlite"'), ..)`) beside a written import of the same database failed everywhere. A site with alternatives or a pattern makes the host's module for each name, as a written site does (src/sema/resolve/code.rs, code/computed.rs, program/code_imports.rs, src/aot/module_keys.rs, load/plugin_modules.rs, and in the engine). 1 test file, 6 cases. - Old, silent, as users build: `const { default: whole } = helper(require("./part.cjs")); whole.default = 5; whole.__esModule = true;`, part.cjs without a marker, and an ES file that imports the default later: `bun <file>` and `--compile` print `number true`, compiled `object false`. A pattern that takes the default of a helper's answer carries the exports object away, and so does a choice on `default` (`||=`, `??=`) (src/aot/modules/commonjs_exports.rs). 49 files of stored programs, 1 crate test. - Old, loud: `require(flag ? "left-out" : "./slow.cjs")` built with `--external left-out`: `bun <file>` and `--compile` answer; --aot: exit 1, "Cannot find module './slow.cjs'". So too with an address in place of the package, with `import()`, beside a module of the library, out of a table. Among alternatives that are written out, a package's name or an address that is left outside no longer ends the walk; a path, a name under a query and a pattern end it as before. And `await import(flag ? "./NOTICE" : "./LICENSE", { with: { type: "text" } })`: exit 1: a file without an ending is a candidate where its name is written out whole and the call names its type (src/sema/resolve/code/computed.rs). 2 test files, 14 cases. - Old, loud: right programs stopped the build: "internal compiler error (Load): the program enters functions of Bun's library that the load has left out": `var i = 0; { function i() {} } [7][i]`, and `/** @jsx list.at */ <a />`. And `import fs = require("node:fs"); { function fs() {} }` in a .cts file: "TypeError: Attempted to assign to readonly property." at the start (src/aot/builtins/resolve/unnamed_uses.rs, mentions.rs, src/sema/bind/runtime.rs and three files below it, program/code_file.rs). 8 stored programs, crate tests. - Old, loud: `const uses = { zlib(found) { .. } }; for (const name in uses) uses[name](require(name));`: exit 1, "Cannot load 'node:zlib': it isn't in this executable". The variable of a `for...in` holds what `Object.keys()` of the object lists, and a call of a member leaves the object as written where every property is a function written there that mentions neither `this` nor `super` (src/sema/import_sites/alternatives.rs). 5 stored programs, crate tests, 1 test file on the time of the build. - Old, test runs with module mocks, an error where `bun test` answers: index.js `export * from "./legacy.js"; export { make } from "./api.js"`, legacy.js `export { make as create } from "./index.js"`, `mock.module("./api.js", () => ({ make: () => "mocked" }))`, reader.js `import { create, make } from "./index.js"`: `mocked,mocked`; compiled "ReferenceError: Cannot access 'create' before initialization". A ring on the way of a name is one name asked twice of one file, not a file met twice (src/aot/lower/module/test_run_ways.rs). 23 files of stored test runs, 1 crate test. - In the engine, the first two with their JavaScript in the messages of oven-sh/WebKit#1000: old, right programs that aborted as users build: a name that a statement re-exports from a CommonJS file that lacks it, beside an `export *` that has it (53 files of stored programs); old, silent, test runs: who was linked under a mock that a second `mock.module()` patches read undefined behind its end (16 files of stored test runs); a build that could end at an assertion, seen in the IR alone. - Old, loud, a right program did not build: `class Counter { static #made = 0; static make(by) { Counter.#made += by; return Counter.#made; } } namespace Counter { export function twice(by) { return Counter.make(by) * 2; } }`: "internal compiler error (ReadAndVerify): StringNotCanonical: string 5 is not canonical WTF-8". One private static member of any kind, a field, an accessor or a method, beside a namespace of the class's name that holds a value. The two share one table of exports, and the private static was listed as a field of the namespace's object. An entry that a class's member with a private name declares is no such field (src/aot/lower/layouts/containers.rs). 13 stored programs, 1 IR test. What Bun without `--aot` does otherwise than before ("main" in this list is main before this commit): This list is not all. A reading of every hunk from main to an earlier commit of the branch outside test/, docs/, src/aot/ and src/js/ahead-of-time/ (627 files, 4,555 hunks) finds 1,592 that a run without --aot passes through: 1,238 in src/sema, which `bun check`, `bun lint`, `bun fmt`, `--check` and `bun build --dts` run and which this list does not name, and 354 elsewhere, 88 separate changes, of which it names nine. The new engine also has eight commits of WebKit's main (oven-sh/WebKit#967 to oven-sh/WebKit#983: 99 files outside aot/, B3 and Air among them) that Bun's main, which pinned 4d45755f, had not shipped. The description of #47942 has the whole list, by folder and file, with what is not by intent and not fixed. - `Bun.build({ compile })` no longer reads `aotHeat`, `aotDisable`, `aotValidate` and `aotLogTypeMismatches`, options of main's compiler. `Bun.build({ compile: { aotHeat: anything } })` always throws now, `compile.aotHeat is not a valid option`: every key of `compile` that begins with `aot` and is not known is refused before any is read (src/runtime/api/JSBundler.rs, src/runtime/cli/compile_options.rs). At the top level `Bun.build({ aotHeat })` threw "goes inside compile" on main and is passed over now; `({ aotOut })` and `({ aotHarden })` throw it now. It reads `aotHarden` and `aotOut`. `packages/bun-types` follows, but for `aotOut`, which is read and not declared. By intent. - Flags of main's compiler that this one does not have are not known any more, among them `--aot-log-type-mismatches`, `--aot-build-test-part` and `--aot-memory-limit`. All begin with `--aot`. By intent. - Bun reads a file of 64 MB or more on several threads, and writes executables the same way. Those threads are named "Bun Parallel"; on main they have no name. A program can see it in /proc/self/task/*/comm. - A `bun test --parallel` worker that waits for its next file sleeps. On main it keeps polling. - Every start of `bun` compares its first argument with the prefix `--aot-program-file`. Nothing follows unless BUN_FEATURE_FLAG_INTERNAL_FOR_TESTING is set. - Native functions whose results are declared to the compiler test a flag before they return. Measured it is 3 to 8 instructions a call in plain Bun: `Blob.text` +3.0 of 1,777, `Blob.slice` +6 of 926, `Blob.type` +8.0 of 342 (exact user instructions a call, Linux x64, a build of main against two builds of the tree that the branch was made from, each read twice; only the +8.0 is laid to the test by the disassembly: the test behind the call takes the thunk's tail call away; the other two are differences of whole builds); about 55 KB of thunks. In a release build the flag is false unless compiled code runs, and a native function that answers outside its declared type answers as before. A build that asserts (debug, `cargo test`, ASAN, or assertions with canary) starts with the flag set: there such an answer throws a TypeError, and a value cached outside its type panics. - In a build of Bun with assertions (debug, ASAN) such a native function throws a TypeError where it answers outside what is declared for it: by intent: it holds the declarations to Bun's own tests. No released build has it as long as no release-safe canary is published: the switch is `Environment::CI_ASSERT` (src/bun_core/env.rs); and a cached value outside what is declared panics there, it throws no TypeError (src/jsc/bindgen.rs). - `bun build --compile --trim` without --aot fills removed code with `hlt` (x64; on macOS `int3`, as main) and `udf` (arm64) where main writes `int3` and `brk`, and also removes what only compiled code enters. By reading; not run. - Each step of a `for await` that drives an async generator passes one test more, whether the process has compiled code. - `bun test`: where the runner loads the next test file while a mock's factory has not settled, that load, which has no importer, keeps its key. Three lines of this branch in `keyOfImportWhileModuleMocksRun()` (src/jsc/bindings/BunPlugin.cpp). By intent. - Two places in the engine's runtime/AbstractModuleRecord.cpp pass one test more, whether the process has compiled code. The items above were held against `git diff` of main and the branch, file by file. What the diff holds beside them is what the reading named at the head of this list is about. Known problems: - A test run under `--isolate` in which an object that one test file made is stored into by the next can crash. - 236 cases in 134 files are marked as known failures, not fixed; each runs every time and turns red the day it passes. 170 of them are bounds on what a program brings of Bun's library and on what `--trim` removes, 15 are on how builds grow, 51 are others. test/aot/goals-not-met-yet.md lists every one with what was measured, the bound, and what keeps the number from growing worse; `bun test/aot/tools/list-goals-not-met-yet.ts` writes that page out of the marks in the tree. - 4 more marks, in 3 files, stand on wrong answers, not goals. Three are on mocks under `bun test --aot`: a mock laid over a file URL reaches the other spelling of the same file too (test/aot/mock-file-url.test.ts, two cases: plain prints `one-importer mocked real`, `two-importers mocked`, `two-importers real`; compiled `mocked mocked`, `mocked`, `mocked`), and a mock that is taken back while a plugin waits is still used (test/aot/unmock-while-loading.test.ts: plain `k runs`, `seen real/slow`; compiled `factory runs`, `seen first/slow`). The fourth is small: where a test catches the error of a file whose import does not parse, `error.name` is `BuildMessage` under `bun test --isolate` and `SyntaxError` under `bun test --aot --isolate` (test/aot/modules-in-a-test-run.test.ts). They say so in their names and stand under a head of their own on the page. - 57 cases in 18 files are declared `test.todo` and never run; two of them, on macOS, stand for faults (test/aot/fault-inside-a-removed-range.test.ts). 5 stored programs are skipped on one platform, because `bun <file>` itself answers now one way, now the other there. - Each found by tests that failed on a build of an earlier commit of the branch: they do not exit and print as `bun <file>` or `bun test` does. Not run again on the last one. Not fixed here; the nine items that follow: - Silent: one data file imported under several `type`s, one of them by a name made at run time (`import("./" + name, { with: { type: "text" } })` beside `import("./conf.yaml", { with: { type: "yaml" } })`): an import can get what another `type` made of the file, or fail as another `type` failed. 7 programs. - Silent, node:http: code of the program that runs while a header's value is looked at comes too late. 2 programs. - Silent, node:http: another object stored as `req._readableState` between a chunk of the body and its event is not looked at; accessors that a program lays above a server, a response or a request are called another number of times than under `bun <file>`. 20 programs. - Silent, streams and EventEmitter: an accessor that a program lays under `emit`, `write`, `push` or `__proto__` is called twice where `bun <file>` calls it once. 6 programs. - Loud: the text of a function that holds JSX, an experimental decorator or `import.meta.glob()`, evaluated again: `SyntaxError: Unexpected token '<'`, `Invalid character: '@'`, `import.meta is only valid inside modules`; `bun <file>` answers. 5 programs. - Loud: `require()` of a conditional name with one written arm like "bun:os" and one unknown arm is refused where `bun run` loads the written arm. 1 program. - Silent, `bun test --aot`: a module that one test file mocks partly (`mock.module("./dep.mjs", async () => ({ ...(await import("./dep.mjs")), extra }))`) hands its state on to a later file that loads it without a mock: `export const list = []`, filled under the mock, is not empty in the next file, where `bun test` runs the module again. 11 test runs. - Loud, `bun test --aot`: a test fails where it passes under `bun test`: two mocks whose async factories import each other's module while a third load leads to both; `await vi.importActual("./a")` where a.ts and b.ts import each other and the factory of a.ts takes the original; `vi.importActual()` inside a factory that is running. 16 test runs. - `bun test --aot`, other shapes of mocks: 81 more test runs in 14 families, among them: a factory that asks for the module that it mocks; a helper that calls `mock.module()` for its caller; a mock laid in a test over the mock of a preload, or over a hoisted mock; a mock laid while the module's body waits at its top-level await; a mock of vitest's kind over a module whose body threw; the entry of `require.cache` of a module that a mock has set aside. - New, loud, as users build, an unusual program: `const uses = { one(found) { .. }, two(found) { .. } }; uses.valueOf``.extra = found => found.name; for (const name of Object.keys(uses)) console.log(name, uses[name](require(`./${name}.cjs`)));`, so too where the tag hands the object to a constant or to `Object.defineProperty()`: `bun <file>` loads all three files, and so did earlier commits of the branch, with the warning TS100064 at the build; now it builds without the warning and ends, after the first two, with "Cannot find module './extra.cjs'", exit 1. A member that is called as a tag is not taken for a call. 3 programs, the first of them run on a build of the branch. Not fixed here. - New, silent, an unusual program: one .sqlite file imported twice under a name made at run time, as `{ type: "text" }` and then as `{ type: "sqlite" }`: `bun <file>`, `--compile` and earlier commits of the branch hand out the first form again (`string string`; the other way round `object object`); compiled, each type gets a form of its own: `string object`, `object string`. 2 programs. Not fixed here. - New, silent, small: a plugin that is added at run time, `build.onResolve({ filter: /\.sqlite$/ }, args => ..)`, is handed the working folder as `args.importer` for a database that is imported under a name made at run time; `bun <file>` and `--compile` hand it the file. Earlier commits of the branch refused that import: "Cannot find module". 1 program. Not fixed here. - Old, silent, as users build: `process.chdir("elsewhere"); const db = (await import(__dirname + "/data.sqlite", { with: { type: "sqlite" } })).default`, so too `path.join(__dirname, "data.sqlite")`: `bun <file>` and an executable of `--compile` open the database beside the file; with --aot the one of the working folder is opened. It shows where the program runs in another folder than that of its file and that folder holds a database of the same name. Not fixed here. - Old, silent, as users build: an import for its side effects (`import "kit/register"`) is removed where a package has a second package.json: the first says `"sideEffects": false` and a nearer one without a name lists the file; or the package's manifest is copied with its list into dist/. 5 programs. Not fixed here. - Old, silent, narrow, as users build: the initializer of a decorated field or accessor (`context.addInitializer()`) reads the value of a parameter property where `bun <file>` has `undefined`: `3 3 1` for `undefined 3 1`. Where it stores into that property, the object keeps the initializer's value and not the argument: `9` for `3`. 2 programs. Not fixed here. - Old, silent, low: a data file of `type: "file"` asked for under a name made at run time in front of the statement that names it is two modules. Not fixed here. - Old, silent, `bun test --aot`: wide.mjs `export * from "./more.cjs"; export * from "./third.cjs"`, last.mjs `export { round } from "./wide.mjs"`, outer.mjs `export { round } from "./last.mjs"`, `vi.doMock("./last.mjs", () => ({ round: "mocked" }))`: `(await import("./outer.mjs")).round` is `5` for `mocked`, and the bodies of both CommonJS files run. Not fixed here. - Old, loud: a script whose class merges with an interface of a file that it names by `/// <reference path="./globals.d.ts" />` stops the build with TS2339 where @types/bun is installed. 1 program. Not fixed here. - Old, loud, as users build: `require(flag ? "./bin/cli" : "./other.js")`, so with `import()`, where ./bin/cli is a file without an ending: "Cannot find module ./bin/cli". 2 programs. Not fixed here. - Old, loud: `module.require()` on the module object of a CommonJS file that threw while it loaded, kept by another file: an error. 1 program. Not fixed here. - Old, loud, a right program does not build: in untyped JavaScript, where a list written out of what a loop left is handed on behind a test, `const list = [left, "x"]; if (..) return target.apply(null, list)`: an operation takes a value of the body that nothing was built for. 1 program. Not fixed here. - Old, loud, a right program does not build: a loop swaps two texts, `const url = new URL(first)` behind it, and a handler reads `url.searchParams`: an operation takes a value of the body that nothing was built for. Not fixed here. - Old, silent, `bun test --aot`, an everyday partial mock: `mock.module("./tax.mjs", async () => ({ ...(await import("./tax.mjs")), rate }))` and then `vi.importActual("./tax.mjs")`: importActual hands out a second instance, whose state is not that of the functions in the mock: `actual 0 real 1 false` for `actual 2 real 3 true`. 18 test runs. Not fixed here. - Old, loud, as users build and in test runs: where some file lets a loader out of sight (`require` handed on, kept in a record, out of `eval("require")`), a file that is passed as another entry point, throws at its first load, is caught and is asked for again: `bun <file>` prints `cold;ran 2;ran 2`; compiled `cold;cold;cold`, or "error: cold", exit 1, where nothing catches. Not fixed here. - Old, loud, `bun test --aot`: a package hands its own `require` on, a test file gives it the path of a CommonJS file of the program, the file throws there and the throw is caught: the written `require("./shaky.cjs")` behind it runs the file again under `bun test`; compiled, the test file ends with the first error, exit 1. Not fixed here. - Old, silent, as users build and in test runs: a data file that is imported plainly and under a type made at run time is two modules under `bun <file>` and one compiled: what is stored through one is read through the other; under "text" the value is the parsed object, not the string. 7 programs. Not fixed here. - Old, silent, as users build: `import "./enter.mjs"; import db from "./data.sqlite" with { type: "sqlite" }`, where enter.mjs calls `process.chdir("b")` at its top level: under `bun <file>` the name meets the folder of the start, with --aot the folder that enter.mjs has changed to. 1 program. Not fixed here. - Old, silent, as users build: `require(file, { type: "text" })`: `bun <file>` gives `{ default: <the text> }`; with --aot the bare text or the parsed value, and of a .cjs file the file runs. 15 programs, and 4 of data files. Not fixed here. - Old, silent, as users build: `createRequire(import.meta.url)("./library")` of a file without an ending that does not parse: `bun <file>` throws; with --aot the text of the file is handed out. 7 programs. Not fixed here. - Old, silent, low, as users build: an entry laid into `require.cache` under the key of a file of the program: `bun <file>` hands it out; with --aot the file runs and answers. 1 program. Not fixed here. - Old, silent, as users build, narrow: what a CommonJS body required before it threw is not among the `children` of its module object: `0 true` for `1 true`. 1 program. Not fixed here. - Old, silent, as users build: a program names `process.mainModule` and runs text that it comes by without naming an evaluator (`f.constructor(text)()`): the text does not find every file in `process.mainModule.children` that it finds under `bun <file>`. 2 programs. Not fixed here. - Old, silent, `bun test --aot` with a preload that mocks a package: a test file ends that mock, lays its own and takes the original: the next test file gets the mock of the preload (`preload` for `real1`). 2 test runs. Not fixed here. - Old, silent, `bun test --aot` with such a preload: the package ran in one test file behind the end of the mock, and the next lays a mock that nobody asks: the third finds it loaded (`listed true real1 bumped 2` for `listed false preload`). 2 test runs. Not fixed here. - Old, loud, `bun test --aot`, narrow: a load is refused because a name is missing, then `vi.doMock()` is laid over the importer: the call itself runs the body of the module and the test fails with a ReferenceError, where `bun test` goes on. 3 test runs. Not fixed here. - Old, silent, `bun test --aot`, no mocks: a test lays `enumerable` on `Object.prototype`; Babel's helper for classes hands `Object.defineProperty()` a descriptor without that word: `bun test` sees the inherited one, the compiled run takes it for absent. Not fixed here. - Old, loud: the build stops with a panic on a ring of `export *` with two CommonJS files and an ambiguous name that is read off the namespace by a written member. Not fixed here. - `bun test --aot`, five kinds. Each was seen on a build of an earlier commit of the branch, against `bun test` by the same binary: 114 small projects in eight families, of which 84 answer otherwise, and two programs more. Not fixed here; the five items that follow: - Old, silent: the original of a mock that takes it (`importOriginal`, a mock without a factory, a spy) runs before the modules that its importer imports in front of it. `import "./setup.js"; import { part } from "./part.js"`, where part.js reads what setup.js sets and a test mocks part.js over its original: `mock real undefined` for `mock real true`. 22 of 24 projects. - Old, silent: once a mock of a CommonJS file has ended, a namespace that passes the file's names on by `export *` shows the names that the mock left out, with the file's values, and the file's body runs: `kept,leftOut 1 2` for `kept 10 undefined`. Where a second `export *` brings such a name too, the namespace loses it: `extra,kept` for `both,extra,kept`. 45 of 70 projects. - Silent: two mocks whose factories wait, loaded at once: a factory that imports the other mocked module gets the mock where `bun test` hands it the file: `mock 1 < mock 2` for `mock 1 < real 2`, and the file's body does not run. A mock laid over a mock whose factory has been called and has not answered reaches the load that began under the old one: `new x | new x | new x` for `old x | old x | new x`. 16 of 20 projects. Not run on older commits. - Old, loud: beside a mock whose factory never answers, a load that does not lead to the mocked module waits for ever: `vi.doMock("./kept.js", () => new Promise(() => {})); try { require("./kept.js") } catch {}; (await import("./side.js")).y` times out where `bun test` prints `real side`. Beside such a mock the `import()` of a module that imports a name which nobody exports waits too, where `bun test` rejects it with a SyntaxError. - Old, silent: `let fail; vi.doMock("./part.js", () => new Promise((resolve, reject) => { fail = reject; })); try { require("./part.js") } catch {}; fail(new RangeError("late"))`: `bun test` reports the RangeError and the test fails; compiled, the test passes. - Old, silent, node:http in an executable: a listener that a program writes into `res._events` is not heard: `close/1` for `close/1,close/2,close/3`; of three listeners of `prefinish` written so, none. 2 programs, on a build of an earlier commit of the branch. Not fixed here. - Reported by a user of the branch; each seen on a build of an earlier commit of this branch, each program built alone by `bun build --compile --aot` and held against `bun <file>`. Not run again on the last one. Not fixed here; the two items that follow: - The build does not end (40 s given): a type alias that grows where it names itself: `type F<T> = { inner: T } & { or?: F<{ inner: T }> }`; `` type F<K extends string> = { key: K; next?: F<`${K}a`> } ``. 6 programs. - Type errors that `tsc` does not report stop the build: TS2322 at `const k: keyof (typeof rows)[number] = "y"` over rows out of a .js file; TS7006 at `getRandomValues: array => array` in a literal of type `Crypto`; TS2416 at `json(): unknown` in a subclass of `Blob` and at a member `count` in a subclass of `Headers`. 4 programs. - Old, loud: what is left of the crash where `addEventListener` or `dispatchEvent` comes out without a receiver (most forms are fixed): the method is read under a name that stands nowhere in the program as a string. 9 programs, each built alone, end with exit 134 ("process.finalization was left out of the executable, because nothing seemed to use it") where `bun <file>` answers. All nine are of the kinds that docs/compiler/compatibility.mdx declares: the name is put together while the program runs (`JSON.parse('["dispatch", "Event"]').join("")`, character codes, `atob()`; read by `signal[name]`, a pattern or a descriptor), or comes out of `for .. in`, of `Object.keys({ dispatchEvent: true })` or of the `name` of a function of the program's; or the method is found without any name, out of `Object.values()` of the prototype. `void dispatchEvent` once anywhere in the program keeps all that the global object leads to. Not fixed here. - Old, silent, `bun test --aot --isolate` and `--parallel`: a source file that a test file writes over behind its first load is read anew by the next test file: `a.test.ts` imports `shared.ts` and writes it over, `b.test.ts` and `c.test.ts` import it: `bun test --isolate`, which keeps the text of every source file that it has read, prints `as built` twice; compiled, `written over` twice. Seen on a build of an earlier commit of the branch. Not fixed here. - New, loud: a program that finds `addEventListener` and `dispatchEvent` by walking `performance` itself and calls them without a receiver: `function both(holder) { let add, send; for (const k in holder) { if (k.length === 16) add = holder[k]; if (k.length === 13) send = holder[k]; } return [add, send]; } const [add, send] = both(performance);`; so `Object.getPrototypeOf(performance)`, `performance.__proto__`, `Object.setPrototypeOf({}, performance)`, `Reflect.get.call(..)`, out of a list, through a constructor. 8 programs, each built alone, answered as `bun <file>` at an earlier commit of the branch and ended with exit 134 at a later one; not run on the last one. `void dispatchEvent` once anywhere in the program keeps all that the global object leads to. Not fixed here. - Old, silent: under `bun test --aot` the factory of `vi.mock()`, `vi.doMock()` or `mock.module()` runs inside the `import()` of the mocked module, where `bun test` runs it in a later job: `let state = "before"; vi.mock("./x", () => { seen = state; .. }); const p = import("./x"); state = "after"; await p;` leaves "before" in `seen` for "after". An import by statement is right. A fix was made on the branch and taken back. Not fixed here. - In an executable `native.dlopen` of bun:ffi is an accessor where plain Bun has a data member: nothing that loads machine code is read while the module loads. Stored as a known failure: test/aot/parallel/what-bun-ffi-hands-out/native-dlopen-is-a-data-member.mjs. Not fixed here. - One script that checks by `Bun.build({ check: true })` and then compiles by `Bun.build({ compile: { aot: true } })` lowers on 32 MB of stack a thread, not 64. No program is known that is compiled otherwise for it; the command line is not touched. Not fixed here. - A loader's name as a label (`case "require":`) beside a read of the global object under an unknown key keeps a program from being closed, so `--trim` is blocked for it. Not fixed here. - In a folder without a tsconfig.json a build stops with TS2339 at `request.body.text()` (also `json()`, `bytes()`, `blob()` of a stream) where the request has a written type `Request`; a tsconfig.json as `bun init` writes it gets past it. Not fixed here. - As docs/compiler/compatibility.mdx says, but silent: under `"useDefineForClassFields": false` one computed key that has to be folded (`["ki" + "nd"] = "user"`) makes all fields of the class defined: `name!: string` erases what the constructor of the base class has assigned (`undefined undefined` for `Ann 3`). Not fixed here. - Silent: `const p = await import("node:process"); Object.keys(p); process.features = 2`: `typeof p.features` is "number" for "object". Not fixed here. - `bun test --aot`: The original of a partial mock that imports another mocked module: `ReferenceError: Cannot access 'o' before initialization` where `bun test` prints `extra,seen mocked`. `mock.module()` called through `forEach()` in a helper mocks nothing, in silence (`real` for `mocked`). Inside a generator of another folder that a spread steps it resolves `./x.mjs` from the folder of the consumer: the run stops (`cannot replace .. because the build didn't see a test mock it`). Not fixed here. - A `.jsonc` file that is no strict JSON (a comment, a comma at the end) beside `import("./" + name, { with: { type: "json" } })`: `bun <file>` throws a SyntaxError at the import; the executable throws `Cannot load .. from a dynamic path`, and the build, which warned of the file before (TS100089), says nothing of it. Loud in both. Not fixed here. - Loud: `process instanceof Probe` where `Probe` has a static `Symbol.hasInstance` that lists the keys of `holder.stdout`: what the method is handed is not seen, and the process ends: `process.stdout .. was left out of the executable`. Not fixed here. - `require(JSON.parse('"./lib"'))` where `lib.js` is a symbolic link and a folder `lib` lies beside it throws MODULE_NOT_FOUND where Bun finds the file. Not fixed here. - Memory, the answers right: a loop that fills a list by `out[at] = at` up to a bound held in a variable asks for the room of the whole bound in front of the loop. Where the loop ends early the room was asked for nothing: a loop to 150,000,000 that throws a RangeError after about thirty turns peaks at 1,195 MB, under `bun <file>` at 23 MB. Not fixed here. - Type errors stop an ahead-of-time build: still a false stop where the build brings Bun's declarations: `timer.toString(36)` on what `setTimeout` returns (TS2554); `case "Linux armv8l":` on `navigator.platform` (TS2678); three cases under `"lib": ["es2019"]`. `// @ts-ignore` gets past each. What the check misses is in the messages on the pull request; two more true errors are known to be forgiven. - `bun test --aot`: odd shapes of mocks (a late object, a file that throws once) answer otherwise than `bun test`; a mock that was taken back while a load waits is used; `vi.doMock()` over an ES module whose body threw (eight shapes answer otherwise, one ends the run); vitest's `importOriginal()` and a mock without a factory over a setup file's mock; an entry planted in `require.cache` makes the next `import()` run the file again; a mock of a barrel does not reach a name handed on from `./inner.js?x`; a helper of the tests that names `Object.prototype` by its name: writes on other built-ins out of sight are refused; under `--rerun-each` what a test file exports stays one object across the runs; `error.name` is "SyntaxError" for "BuildMessage" under `--isolate`. - A store into a typed array at a negative index is slow; `new this.Proxy` in a sloppy function; the old spelling of a path above the root written out by hand. - A plugin that no compiled file names, with code from disk; two sibling modules that both await; late private symbols of node:http; text run by `eval` that reads a member of `process` which the build left out; an `onResolve` callback that throws at a later name; `routes: { "/*": { dir, style } }` is refused; an accessor under "noDeprecation" or "throwDeprecation" on `EventEmitter.prototype` ends the process; a folder named `root` beside the folder of the entry point; the bytes of an executable hang on whether a file named as the entry point's stem lies beside it; accessors on the own names of a request or a socket, or on `IncomingMessage.prototype`, that throw once; `const f = true && function () {}`: `f.name` is right in 9 of 12 forms. - Builds that grow faster than their programs: loops nested hundreds deep (a square); two rows of classes whose tables grow by the square; n classes below one base and an object of them out of sight (four times the build for twice the classes); N calls of a helper with a loop in one straight block (a square); the members of one literal with as many objects created below it (a square); a barrel of k `export *` with one `vi.doMock()` on one of its files (7.2, 20.0 and 67.6 G instructions for 1,000, 2,000 and 4,000 files); a type that names the one before it twice, `type M2 = Map<M1, M1>` and so on down, at a declared field of a class, its initializer, a parameter property or a value handed to a function of a JavaScript file (twice the build for each level). - New, loud, as users build, an unusual program: `const uses = { a(found) { return found.v; } }; uses.valueOf().b = found => found.v + 1; for (const name in uses) console.log(name, uses[name](require(`./rows/${name}.cjs`)));`, so too with `uses.__defineGetter__("b", ..)`: `bun <file>` prints `a 1`, `b 3`, and so did earlier commits of the branch with --aot, the last behind a warning of the build (an executable of `--compile` does where ./rows lies beside it, and else ends with an error); now it builds without a warning and prints `a 1` and "Cannot find module './rows/b.cjs'", exit 1. An inherited method sees its object and can change it; the walk that follows a name out of the keys of a written object does not ask which member is called. So too where the object is added to by a second name, `const uses = module.exports = { .. }; module.exports.extra = ..` or `const uses = other = { .. }; other.extra = ..`, by another file through what the file exports, `part.table.extra = ..`, by a direct `eval` in a method of such an object in a list that is walked, or by another of the names that `forEach()` over a Set written in place, `find()` or `filter()` gives it: "Cannot find module './extra.cjs'", exit 1. Where the names are built-in modules the warning alone is lost. Not fixed here. - New, loud, `bun test --aot`: where a written object gains a key through `__defineGetter__` and a test deletes the entries of `require.cache` whose paths are made of its names in `for (const name in made)`, the test run ends with exit 1 and says nothing; `bun test` passes. 1 test run. Not fixed here. - Old, loud: `var uses = { zlib(found) { .. } }; for (const name in uses) uses[name](require(name));`: exit 1, "Cannot load 'node:zlib': it isn't in this executable"; with `const` and `let` it is the item on the `for...in` above. So too `var uses; uses = { .. }`, `var names = Object.keys(uses); for (const name of names) ..`, and keys listed by `Object.getOwnPropertyNames()`, `Reflect.ownKeys()` and `Object.entries()`. Not fixed here.
Follow-up to oven-sh#47316 for `bun build --compile --aot` and `bun test --aot`. Goes with oven-sh/WebKit#967; WEBKIT_VERSION is pinned to its preview build until it merges. bun test --aot: - Tests can replace `Math`, `JSON`, `Reflect` or `process` on the global object: `global.Math = mockMath`, `globalThis.process = undefined`. - Tests can pose as another runtime: redefining or assigning `process.versions`, `process.release`, `process.isBun`. - `swap(Date, { now() { return 5 } })`, where a helper in another file copies the members in a `for...in` loop, no longer fails with "Cannot assign to Date.now". - In layered runs, built-ins reached through another file stay writable: `export { getPrototypeOf } from "./tools.js"`, `export const above = x => Object.getPrototypeOf(x)`, a static field, a `Map`. - Names that `mock.module()` adds or leaves out are seen through `export *`. - A `mock.module()` factory that throws rejects the import the way `bun test` does, also for a module that is not on disk. Module loading: - `require.cache[key].exports` of a file required as a statement is its exports, not `undefined`. The entry of an ES module shows all of its exports. - `require.cache[key].children` and `.parent` are filled for files required at the top level of a test file. - Imports with nothing to run (`export const value = 7`) are listed in `Object.keys(require.cache)`. - `require("./config.json")` in a CommonJS file returns the data while an ES module that imports the same file is still loading. It returned the ES namespace. - `const name = "bun"; (await import(name)).default === Bun` answers what Bun answers. Crashes: - `mock.module("./below.ts", () => { throw .. })` followed by imports of two files above it aborted in builds with assertions. - When the source text an error message quotes failed its own check, the build stopped with an internal compiler error. Now that one file loses the quote. - `function kind(x) { switch (x) { case 3: .. } } kind([2])` aborted with "FrameCellEscaped" in builds with validation on. Wrong answers: - `function pick(i) { return i % 2 ? null : { a: 1 } } function read(o) { return o.a }`: `read(null)` returned 1 instead of throwing. - ES5 classes that share a method: after `delete A.prototype.twice`, `new A(8).twice()` returned 16 instead of throwing. - `try { null.x } catch (e) { e.constructor.extra = 41 }`: `TypeError.extra` was `undefined`. - `Object.getPrototypeOf(crypto.subtle).extra = 41`: `SubtleCrypto.prototype.extra` was `undefined`. Same for `AbortSignal.prototype` and `Bun.S3Client.prototype`. - `class Tags extends Set { constructor(values) { super(values) } }` calls an overridden `add`. Same for `Map`, `WeakMap`, `WeakSet`. - `Array.from(new Map(..))` after the class's iterator was deleted treats the object as array-like. - `Object.assign({}, source)` kept the wrong key order for a source with a hidden index property and a symbol key. Faster: - Classes that Babel compiled to ES5 are compiled as classes: `_createClass`, `_defineProperties`, `_classCallCheck`, `_inherits` and `_typeof`, written into the file or imported from `@babel/runtime`, Babel 7.18 through 7.24. - `var p = Object.create(A.prototype); p.constructor = B; B.prototype = p`: 254 to 25 instructions per iteration. - `util.inherits = function (ctor, superCtor) {..}` in another file: 712 to 19. - `const f = table[key]; for (..) t += f(..)` checks once, before the loop, which function it got. - `out[key] = value` that adds a property is cached per site. `holder[key] = value` checks the field's type once per kind of value. - `for (const [key, value] of [["a", 1]])` and `const [text, count] = ["abc", n]`: each position has its own type. Smaller: - The names of captured variables are written once for closures that capture the same ones. - The registers around a slow-path call are saved by one shared stub. A caller that reads no floating-point register after a call saves none. Build time and memory: - `let picked = s0; if (n > 1) picked = s1; ..` with 2,000 or more branches. - A linked package reached through a computed `import()` no longer causes a second build. Tests and CI: - A test builds each stored program under several orders of the analysis rules, thread counts and checker passes, and fails if the generated code differs. - The instruction counts in `test/aot/perf` are exact, with no tolerance. Not AOT-specific: - `bun test --isolate`: a `process.nextTick()` queued while the second test file's imports load runs after that file's body again, as before oven-sh#47316. - `bun test --parallel`: a worker waiting for its next file sleeps instead of polling. - `bun test`: the memory reservation while tests run is removed (581 lines). - `bun test --isolate`: a `FileSink` closes its descriptor before `close` returns instead of later on the work pool. - `require.cache` is the function it was before the first AOT PR. The traps for compiled programs are a separate function. - `bun check`: a hash set past 16 type parameters, the narrowing index is kept instead of dropped whole, the regular form of a literal is computed once, a discarded task explains no error. - `path.posix.dirname`, `basename` and `extname` of Latin-1 strings use SIMD kernels.
Follow-up to #942 for
bun build --compile --aotandbun test --aot. The Bun half, with the tests that need Bun, is a PR in oven-sh/bun that pins this one's preview build.Crashes of compiled programs
Eleven lines ended by SIGSEGV, and ten lines of the same kind aborted the compiler:
SinkAllocationsanalyses a function, counts, then rewrites it, and all three steps must decide alike. A helper read constants out of the function as it stood, so the rewriting saw constants that the analysis hadn't. AOT: per-global program state, mockable exports, heap-allocated generator frames, bug fixes #942 fixed one of its five callers. Now the question is answered from a table filled before anything is rewritten, and that fix is gone again.try { null.x } catch (e) { e.constructor.list = { length: 3 } }, whereTypeError.listwas an array before: a signal,undefined, or a call of the function that had been replaced. Flow analysis took the stores it could follow for all the stores there are. A store through an object nobody knows is now looked for on every built-in class.Bun.gzipSync.list = [1, 2]; Bun[JSON.parse('"gzipSync"')].list = { length: 3 }, thenBun.gzipSync.list.length: SIGSEGV. The same throughEvent.prototype[name], names out ofObject.keys(Bun), andBun.Glob.prototypereached by three computed names. A read under a computed key, or ofprototype, off an unknown object in the program's own code now reaches the host's prototype objects.A loop counter beyond int32, read from a property and used to index a typed array, trapped.
class Holder { held = stream[Symbol.for("nodejs.stream.readable")]; }with aReadableStreamaborted. In a function the read gaveundefined. A symbol fromSymbol.for()was treated like one fromSymbol(), which the host can't reach.Crashes of the compiler
while (a == 65536 && i0 < 9) { i0++; c = c > 0 ? d : c; }.B3VectorizeLoopsdivided by the number of memory accesses of a loop that has none.bun test --aoton NestJS's own suite stopped with "Stopped while compiling printMessages .. in PlaceTypeTests". So did tslog's (in webpack's_getRenamedInlineModule) and a project that uses knex. A block's parameters are its explicit arguments followed by the results of the edges that enter it, such as the narrowed value of a type test. Two places indexed an edge's arguments with a parameter's index, which is out of bounds for an edge's own result. An earlier commit of this PR brought it.Wrong answers
/^(?=(.*))\1$/said "no match" for a subject with characters outside Latin-1. The pattern was taken to consume 8-bit characters only. What a lookahead captures counts once a back reference can consume it.new RegExp("\\B" + text, "u"):\Bmatched inside a surrogate pair.function make(k) { return k === 0 ? new A(0) : new B(1) }with a getter onB.prototypealone:make(0).sizeranB's getter when the firstAwas made behind a test of both classes.x.f += nwherefstarts on the prototype: the arm that readsfby name may run a getter, and it came back into code that relied on a guard from before the read.Interface.prototype[Symbol.asyncIterator]read in a function was a different function object thanReflect.get()returned (node:readline).await import("sass")in an executable threw "Cannot assign to Function.prototype.call$0". Programs out of dart2js add such names.function twice(f, n) { return f(f(n)) } twice({}, 1): the error message quotedf, notf(n).delete A.prototype.twice,new A(8).twice()still answered.B.prototypealone,const o = k === 0 ? new A(k) : new B(k); o.y = 5in a loop ranB's setter for the firstA. B3 moved the load of a class's guard bits across the allocation that makes the class.require("./config.json")in a.cjsfile that runs while the graph loads answered the ES namespace.Object.assign({}, source)stored names before symbols for a source with room for elements.o.awas folded whereomay benull, and another pass aborted the build.Memory
""or of a one-character Latin-1 string needs no write barrier.for (let i = 0; i < n; i++) a.push(f(i))reserved the whole array up front even wherefmay throw and end the loop.function ask_i(x) { return x.weigh() + ask_(i-1)(x) }, 800 in a chain, each given an instance of one class more. No two calls ofweighsee the same classes, so the cache kept a partition of the receivers per call and key: 320,000 arms of 112 bytes. A partition is now kept packed, 24 bytes for an arm with one method and one layout, and "may this call enter a table look-up" asks class by class and keeps nothing. 189 > 166 MB, with identical output on 1.16 million functions.Build time
A generated function of 1,000 statements builds in 7,891 M instructions, down from 12,473 M, with identical output:
test/aot/build-scalingin Bun gets its first shapes with a warm body.Speed
a[i] === b[j].let current = first; while (current) current = current.run(this)withfirsttyped by an interface: objects that have the interface's members but none of its layouts stay in the typed loop. Effect's run loop takes 20% fewer instructions per turn.TypeErrorat a place quotes the source ("undefined is not an object (evaluating 'a.b')"). For that the run time decoded every entry of the kept piece of source at each first error, and a long statement is one piece. Now once per piece. Forty places in oneswitchof 12,000 cases: 2.3 > 1.5 ms, in a compiled test run 4.05 > 0.59 ms. No message changes.Windows
hasChanged |= std::exchange(member, copyOfClass(member)) != membercompares the old value with itself where the right operand is evaluated first, as on both Windows targets. Copied classes and closure factories kept the original's types there: correct, but less precise.Size
Checks
--aot-check=ClaimsOfFlowtests every claim of flow analysis where it's made.SinkAllocationschecks that its three steps decided alike.RegExp::matchWithImageCodedereferenced a null image record when jsc runs its tests with regular expressions compiled ahead of time. That was six failing JSTests on this PR's first commit.--aot-reportsay "proven" or "missed optimization" where the final IR knows the receiver's class, not "unknown".Changes outside
aot/22 files:
b3/andb3/air/(branches to selects, loop rotation, the loop vectorizer, the greedy register allocator, block order,AirReleaseDeadValues, threetestb3files),runtime/(ArrayPrototype,JSModuleLoader,JSObject,RegExp),yarr/(YarrJIT,YarrFlatProgram) andwtf/FrequencyEstimator.h.